Category Archives: Programming

“Mastering AngularJS Directives” (Book Review)

Unlike many general introduction books, “Mastering AngularJS Directives” by Josh Kurz takes a much more specialized approach. It assumes you know AngularJS fairly well and explores just one (but arguably the most complex) of its corners: directives.

It’s not a thick book and the table of contents looks just right: Basic introduction to directives, a simple example, and then digging deeper into integration of third party libraries, compilation, communication between directives, writing directives to watch live data for changes, and finally some optimization and code quality notes.

Unfortunately, the book is rather poorly written. It is confusing even to someone who has been using AngularJS profesionally for over 1.5 years. The explanations tend to be short and often miss the point. You may see a difficult issue brought up, followed by a listing over 2 pages long, and finally left with unsatisfactory explanation of how it works or why you would do it this way. In some ways it just lacks focus.

There are some substantive errors too – calling JS objects “JSON notation”, mentioning singletons giving you a new instance every time etc.

That said, even though it is a difficult read, it is not without value. I learned quite a few things myself, some of them mentioned directly and some between the lines. It’s one of the first attempts at thorough introduction to directives and it still may come in handy at times.

The bottom line – I am not sure if I would recommend it to a friend. I liked ?Mastering Web Application Development with AngularJS? by Paweł Kozłowski and Peter Darwin a lot better, and even though it’s not dedicated to directives it does better job at explaining them.

Navigation and Routing with Om and Secretary

After some quick experiments with Secretary and Enfocus, I decided to dive headfirst to Om.

Since I’m kind of restarting my pet project all the time, the first thing I lay down is routing and navigation. This time I’ll implement it by combining Secretary with Om and a little Bootstrap.

One of the key features of Om is strong separation of state from behavior from rendering. In a nutshell, state is defined in one place in an atom and is just, you know, state. You can manipulate it as you like without worrying about rendering. Finally, you install renders on top of it without worrying about the behavior.

Let’s start with a bunch of imports. We’ll need Secretary and goog.History from Closure as well as some Om for rendering. I’ll also keep a reference to History so I don’t instantiate it over and over.

(ns demo.navigation
  (:require [secretary.core :as secretary :include-macros true :refer [defroute]]
            [goog.events :as events]
            [om.core :as om :include-macros true]
            [om.dom :as dom :include-macros true])
  (:import goog.History
           goog.History.EventType))

(def history (History.))

Now, the state. Each route has a name that will appear on the navigation bar and path for routing.

(def navigation-state 
  (atom [{:name "Add" :path "/add"}
         {:name "Browse" :path "/browse"}]))

Time for some state manipulation. Enter Secretary and Closure history:

(defroute "/add" [] (js/console.log "Adding"))

(defroute "/browse" [] (js/console.log "Browsing"))

(defn refresh-navigation []
  (let [token (.getToken history)
        set-active (fn [nav]
                     (assoc nav :active (= (:path nav) token)))]
    (swap! navigation-state #(map set-active %))))

(defn on-navigate [event]
  (refresh-navigation)
  (secretary/dispatch! (.-token event)))

(doto history
  (goog.events/listen EventType/NAVIGATE on-navigate)
  (.setEnabled true))

It’s very similar to what I did before – two basic routes, gluing Secretary to Closure history with pretty much the same code that is in Secretary docs.

There’s one thing worth noting here. Every time the route changes, refresh-navigation will update the navigation-state atom. For each of the routes it will set the :active flag, making it true for the path we navigated to and false for all others. This will be used to render the right tab as active.

Now, somewhere in my HTML template I’ll put the div to hold my navigation bar:

<div id="navigation"></div>

Finally, let’s do the rendering in Om:

(defn navigation-item-view [{:keys [active path name]} owner]
  (reify
    om/IRender
    (render [this]
            (dom/li #js {:className (if active "active" "")}
                    (dom/a #js {:href (str "#" path)} name)))))

(defn navigation-view [app owner]
  (reify
    om/IRender
    (render [this]
            (apply dom/ul #js {:className "nav nav-tabs"}
                   (om/build-all navigation-item-view app)))))

(om/root navigation-view navigation-state
         {:target (. js/document (getElementById "navigation"))})

Let’s investigate it from the bottom.

om/root binds a component (navigation-view) to state (navigation-state) and installs it on the navigation element in DOM.

navigation-view itself is a composite (container) component. It creates a <ul class="nav nav-tabs"> containing a navigation-item-view for each route.

Finally, navigation-item-view renders <li class="active"><a href="#{path}">{name}</a></li> using the right pieces of information from the map representing a route.

That’s it. Like I said, state is as pure as it can be, routing doesn’t know anything about rendering, and rendering only cares about state. There is no explicit call to rerender anything anywhere. What’s more, reportedly Om is smart enough to figure out exactly what changed and keep the DOM changes to minimum.

Side note – Om looks like a big thing to learn, especially since I don’t know React. But it’s quite approachable thanks to its incredibly good tutorial. It also made me switch from Eclipse with CounterClockWise to LightTable, giving me more productive fun than I can remember.

“Clojure Cookbook” by Luke VanderHart, Ryan Neufeld; O’Reilly Media

Clojure Cookbook

O’Reilly has just published a new book on Clojure, this time from the “cookbook” series. The book includes over 150 practical recipes on doing some common things in Clojure. Each recipe is self-contained and usually very small.

It starts with a detailed walkthrough of primitive and collection manipulations. Then it includes recipes on basic development tasks (REPL, using docs, running programs etc.), I/O, databases (two recipes on SQL, one for each of a handful of NoSQL databases, plus quite a few on Datomic), web applications with Ring, performance optimization, distributed computing (mostly Cascalog, some Storm) and testing.

In my opinion the book is very uneven. It’s very detailed about the primitives and basic collections, but at the same time it doesn’t do justice to state management (atoms, refs, agents) or concurrency. Yet it has two chapters on building a red-black tree. It is very detailed about Datomic, but barely scratches the surface of much more common tools like core.async, core.logic or core.match. It does not include anything about graphics or ClojureScript.

In short, it sometimes pays much attention to some uncommon problems or tools, while giving too little information on more popular pieces. I think the target audience is somewhere around intermediate. I don’t think it’s a good way to get started with the language, but it is a decent, handy survey of some areas of the landscape.

Careful With Native SQL in Hibernate

I really like Hibernate, but I also don’t know a tool that would be nearly as powerful and deceptive at the same time. I could write a book on surprises in production and cargo cult programming related to Hibernate alone. It’s more of an issue with the users than with the tool, but let’s not get too ranty.

So, here’s a recent example.

Problem

We need a background job that lists all files in a directory and inserts an entry for each of them to a table.

Naive Solution

The job used to be written in Bash and there is some direct SQL reading from the table. So, blinders on and let’s write some direct SQL!

for (String fileName : folder.list()) {
    SQLQuery sql = session.getDelegate().createSQLQuery(
        "insert into dir_contents values (?)");
    sql.setString(0, fileName);
    sql.executeUpdate();
}

Does it work? Sure it does.

Now, what happens if there are 10,000 files in the folder? What if you also have a not so elegant domain model, with way too many entity classes, thousands of instances and two levels of cache all in one context?

All of a sudden this trivial job takes 10 minutes to execute, all that time keeping 2 or 3 CPUs busy at 100%.

What, for just a bunch of inserts?

Easy Fix

The problem is that it’s Hibernate. It’s not just a dumb JDBC wrapper, but it has a lot more going on. It’s trying to keep caches and session state up to date. If you run a bare SQL update, it has no idea what table(s) you are updating, what it depends on and how it affects everything, so just in case it pretty much flushes everything.

If you do this 10,000 times in such a crowded environment, it adds up.

Here’s one way to fix it – rather than running 10,000 updates with flushes, execute everything in one block and flush once.

session.doWork(new Work() {
    public void execute(Connection connection) throws SQLException {
        PreparedStatement ps = connection
                .prepareStatement("insert into dir_contents values (?)");
        for (String fileName : folder.list()) {
            ps.setString(1, fileName);
            ps.executeUpdate();
        }
    }
});

Other Solutions

Surprise, surprise:

  • Do use Hibernate. Create a real entity to represent DirContents and just use it like everything else. Then Hibernate knows what caches to flush when, how to batch updates and so on.
  • Don’t use Hibernate. Use plain old JDBC, MyBatis, or whatever else suits your stack or is there already.

Takeaway

Native SQL has its place, even if this example is not the best use case. Anyway, the point is: If you are using native SQL with Hibernate, mind the session state and caches.

ClojureScript Routing and Templating with Secretary and Enfocus

A good while ago I was looking for good ways to do client-side routing and templating in ClojureScript. I investigated using a bunch of JavaScript frameworks from ClojureScript, of which Angular probably gave the most promising results but still felt a bit dirty and heavy. I even implemented my own routing/templating mechanism based on Pedestal and goog.History, but something felt wrong still.

Things have changed and today there’s a lot buzz about React-based libraries like Reagent and Om. I suspect that React on the front with a bunch of “native” ClojureScript libraries may be a better way to go.

Before I get there though, I want to revisit routing and templating. Let’s see how we can marry together two nice libraries: Secretary for routing and Enfocus for templating.

Let’s say our app has two screens which fill the entire page. There are no various “fragments” to compose the page from yet. We want to see one page when we navigate to /#/add and another at /#/browse. The “browse” page will be a little bit more advanced and support path parameters. For example, for /#/browse/Stuff we want to parse the “Stuff” and display a header with this word.

The main HTML could look like:

<!DOCTYPE html>
<html>
<body>
	<div class="container-fluid">
		<div id="view">Loading...</div>
	</div>

	<script src="js/main.js"></script>
</body>
</html>

Then we have two templates.

add.html:

<h1>Add things</h1>
<form>
  <!-- boring, omitted -->
</form>

browse.html:

<h1></h1>
<div>
  <!-- boring, omitted -->
</div>

Now, all we want to do is to fill the #view element on the main page with one of the templates when location changes. The complete code for this is below.

(ns my.main
  (:require [secretary.core :as secretary :include-macros true :refer [defroute]]
            [goog.events :as events]
            [enfocus.core :as ef])
  (:require-macros [enfocus.macros :as em])
  (:import goog.History
           goog.History.EventType))

(em/deftemplate view-add "templates/add.html" [])

(em/deftemplate view-browse "templates/browse.html" [category]
  ["h1"] (ef/content category))

(defroute "/" []
  (.setToken (History.) "/add"))

(defroute "/add" []
  (ef/at 
    ["#view"] (ef/content (view-add))))

(defroute "/browse/:category" [category]
  (ef/at 
    ["#view"] (ef/content (view-browse category))))

(doto (History.)
  (goog.events/listen
    EventType/NAVIGATE 
    #(em/wait-for-load (secretary/dispatch! (.-token %))))
  (.setEnabled true))

What’s going on?

  1. We define two Enfocus templates. view-add is trivial and simply returns the entire template. view-browse is a bit more interesting: Given category name, alter the template by replacing content of h1 tag with the category name.
  2. Then we define Secretary routes to actually use those templates. All they do now is replace content of the #view element with the template. In case of the “browse” route, it passes the category name parsed from path to the template.
  3. There is a default route that redirects from / to /add. It doesn’t lead to example.com/add, but only sets the fragment: example.com/#/add.
  4. Finally, we plug in Secretary to goog.History. I’m not sure why it’s not in the box, but it’s straightforward enough.
  5. Note that in the history handler there is the em/wait-for-load call. It’s necessary for Enfocus if you load templates with AJAX calls.

That’s it, very simple and straightforward.

Update: Fixed placement of em/wait-for-load, many thanks to Adrian!

“Version Control with Git, 2nd Edition” by Jon Loeliger, Matthew McCullough; O’Reilly Media

Version Control with Git

There are reasons why Git has become so popular, but the first encounter with it can a bit overwhelming. Even if you kind of learn how to do basic things, it’s not uncommon to feel like we’re only scratching the surface. The typical reaction when something slightly less typical is needed often sounds like: “There be dragons!”

Here comes “Version Control with Git” by Jon Loeliger and Matthew McCullough.

It starts with a good explanation of the basic concepts of Git. It explains all the building blocks of Git and internal organization of repository. It slowly introduces the basic commands and every time explains very well how a change is reflected in the repository or what a command is really operating on.

The distribution, collaboration, merging etc. are introduced fairly late, but somehow by that time the reader will have understood the core so much that everything just falls into place and is immediately understandable. Finally, it also shows some more arcane features and commands that probably are rarely used, but knowing that they are there and having the book handy for when the time comes doesn’t hurt.

Last but not the least, it explains common usage patterns as well as things that can be done outside the typical path, with appropriate warnings about possible negative impact.

This book is a must-read for all Git users. It’s usable on all levels, from absolute newbie to someone who feels fairly proficient with Git. I’ve been using Git daily for quite a while, and it really helped me understand what is going on. Everything is very accessible, with plenty of examples as small and practical as possible, as well as some images.

Direct Server HTTP Calls in Protractor

When you’re running end-to-end tests, chances are that sometimes you need to set up the system before running the actual test code. It can involve cleaning up after previous executions, going through some data setup “wizard” or just calling the raw server API directly. Here’s how you can do it with Protractor.

Protractor is a slick piece of technology that makes end-to-end testing pretty enjoyable. It wires together Node, Selenium (via WebDriverJS) and Jasmine, and on top of that it provides some very useful extensions for testing Angular apps and improving areas where Selenium and Jasmine are lacking.

To make this concrete, let’s say that we want to execute two calls to the server before interacting with the application. One of them removes everything from database, another kicks off a procedure that fills it with some well-known initial state. Let’s write some naive code for it.

Using an HTTP Client

var request = require('request');

describe("Sample test", function() {
    beforeEach(function() {
        var jar = request.jar();
        var req = request.defaults({
            jar : jar
        });

        function post(url, params) {
            console.log("Calling", url);
            req.post(browser.baseUrl + url, params, function(error, message) {
                console.log("Done call to", url);
            });
        }

        function purge() {
            post('api/v1/setup/purge', {
                qs : {
                    key : browser.params.purgeSecret
                }
            });
        }

        function setupCommon() {
            post('api/v1/setup/test');
        }
        
        purge();
        setupCommon();
    });

    it("should do something", function() {
        expect(2).toEqual(2);
    });
});

Since we’re running on Node, we can (and will) use its libraries in our tests. Here I’m using request, a popular HTTP client with the right level of abstraction, built-in support for cookies etc. I don’t need cookies for this test case – but in real life you often do (e.g. log in as some admin user to interact with the API), so I left that in.

What we want to achieve is running the “purge” call first, then the data setup, then move on to the actual test case. However, in this shape it doesn’t work. When I run the tests, I get:

Starting selenium standalone server...
Selenium standalone server started at http://192.168.15.120:58033/wd/hub
Calling api/v1/setup/purge
Calling api/v1/setup/test
.

Finished in 0.063 seconds
1 test, 1 assertion, 0 failures

Done call to api/v1/setup/purge
Done call to api/v1/setup/test
Shutting down selenium standalone server.

It’s all wrong! First it starts the “purge”, then it starts the data setup without waiting for purge to complete, then it runs the test (the little dot in the middle), and the server calls finish some time later.

Making It Sequential

Well, that one was easy – the HTTP is client is asynchronous, so that was to be expected. That’s nothing new, and finding a useful synchronous HTTP client on Node isn’t that easy. We don’t need to do that anyway.

One way to make this sequential is to use callbacks. Call purge, then data setup in its callback, then the actual test code in its callback. Luckily, we don’t need to visit the callback hell either.

The answer is promises. WebDriverJS has nice built-in support for promises. It also has the concept of control flows. The idea is that you can register functions that return promises on the control flow, and the driver will take care of chaining them together.

Finally, on top of that Protractor bridges the gap to Jasmine. It patches the assertions to “understand” promises and plugs them in to the control flow.

Here’s how we can improve our code:

var request = require('request');

describe("Sample test", function() {
    beforeEach(function() {
        var jar = request.jar();
        var req = request.defaults({
            jar : jar
        });
        
        function post(url, params) {
            var defer = protractor.promise.defer();
            console.log("Calling", url);
            req.post(browser.baseUrl + url, params, function(error, message) {
                console.log("Done call to", url);
                if (error || message.statusCode >= 400) {
                    defer.reject({
                        error : error,
                        message : message
                    });
                } else {
                    defer.fulfill(message);
                }
            });
            return defer.promise;
        }
		


        function purge() {
            return post('api/v1/setup/purge', {
                qs : {
                    key : browser.params.purgeSecret
                }
            });
        }

        function setupCommon() {
            return post('api/v1/setup/test');
        }
		
        var flow = protractor.promise.controlFlow();
        flow.execute(purge);
        flow.execute(setupCommon);
    });

    it("should do something", function() {
        expect(2).toEqual(2);
    });
});

Now the post function is a bit more complicated. First it initializes a deferred object. Then it kicks off the request to server, providing it with callback to fulfill or reject the promise on the deferred. Eventually it returns the promise. Note that now purge and setupCommon now return promises.

Finally, instead of calling those functions directly, we get access to the control flow and push those two promise-returning functions onto it.

When executed, it prints:

Starting selenium standalone server...
Selenium standalone server started at http://192.168.15.120:53491/wd/hub
Calling api/v1/setup/purge
Done call to api/v1/setup/purge
Calling api/v1/setup/test
Done call to api/v1/setup/test
.

Finished in 1.018 seconds
1 test, 1 assertion, 0 failures

Shutting down selenium standalone server.

Ta-da! Purge, then setup, then run the test (again, that little lonely dot).

One more thing worth noting here is that control flow not only takes care of executing the promises in sequence, but also it understands the promises enough to crash the test as soon as any of the promises is rejected. Once again, something that would be quite messy if you wanted to achieve it with callbacks.

In real life you would put that HTTP client wrapper in a separate module and just use it wherever you need. Let’s leave that out as an exercise.

“RESTful Java with JAX-RS 2.0, 2nd Edition” by Bill Burke; O’Reilly Media

RESTful Java with JAX-RS 2.0

REST is all the rage now (not without a reason), and in the Java world the standard API for that is JAX-RS (under the JEE umbrella). “RESTful Java with JAX-RS 2.0” is the second edition of Bill Burke’s book on the JAX-RS API. Bill Burke is the creator of RESTEasy and a member of the committee that designed JAX-RS.

The book is divided into two parts, over a dozen short chapters in each. The first part includes a very nice introduction to REST, has a great systematic reference over the API and finally a few words on integration with various frameworks, security, caching etc. The second chapter is basically a workbook – there is downloadable code with a few examples for each chapter, and these chapters basically are a detailed walk through.

The author is careful not to get ahead of himself and starts quite slow, introducing the more advanced, automated and magical features step by step. As a result, the book is a great introduction for complete newcomers. But it doesn’t stop there – it discusses all the more advanced features as well (later), with the same depth and clarity. That makes it a great reference and a cookbook that you’re likely to get back to as use the API in your work.

Everything is well thought out and executed. It’s a very easy read, with each chapter stating the problem that it’s trying to solve, following up with presentation of the relevant part of the API and a number of practical examples. If for some reason you need more, you’re free to explore the workbook or complete running code.

Highly recommended.

Note on edition: I read it on Kindle, no issues at all.

“Mastering Web Application Development with AngularJS” (Book Review)

While the first demos and tutorials of AngularJS make very good impression, using it on your own in real life applications quickly leads to confusion and frustration. You soon discover that the documentation falls short of explaining what really is going on, especially in the more advanced areas. It does not do a very good job at showing idiomatic usage either – with proper separation of responsibilities, use of services and directives, etc.

“Mastering Web Application Development with AngularJS” by Paweł Kozłowski and Peter Darwin is really good resource to fill those gaps. It starts with a decent explanation of what AngularJS is all about. How DOM is some kind of a skeleton behind the application, or in other words how application state is directly reflected in DOM. Right after this introduction it introduces unit testing, and from this point on everything is demonstrated not only with the “production” code, but also with accompanying test suites.

Then it starts to dig a bit deeper – from filters, communcation with back-end and navigation through writing custom directives and performance. While the beginning seems to be a bit slow, the chapters on directives are really detailed, have plenty of great examples and do an outstanding job at explaining this difficult subject. Actually, I would say that the whole book may be a bit too advanced for beginners, but then even if you have some experience with Angular, it is well worth reading for the directives alone.

The entire book is organized as a systematic “reference”, with each chapter dedicated to one aspect of the framework: Binding and filters, communication with server, forms, navigation and routing, directives, internationalization, build/deployment, and so on. There also is a complete non-trivial application available on Github, and referenced throughout the book. Each and every aspect has a very accessible and complete explanation. Theory and rationale, working code as well as test suites.

In other words, the book is not a simplistic tutorial, but a detailed study that takes a reasonably complex application and dissects it one “dimension” at a time. You don’t need to study the entire application while reading the book, but it’s a great complementary material that demonstrates how the pieces fit together and is a ready-to-use cookbook of some sort.

If there is anything missing, I would say it’s information on idiomatic usage: How you are supposed to structure your application, divide it into modules and services, and so on. Not that it’s completely missing from the book, but a bit of a bird’s-eye view would be nice as well.

All in all, it’s definitely worth reading. Detailed, non-trivial, doing a great job at explaining the “why’ and demonstrating the “how”.

(I got the book directly from Packt and read it on Kindle – nothing to complain about in this edition, everything readable and comprehensible.)

The future may just as well be RESTful

Chris Zheng has just published an article on “Why the future is NOT RESTful”. It made a bit of a splash, but I think it’s based on false assumptions and quite wrong. Here’s why.

Chris observes that client takes more and more responsibilities from the server. He suggests that server is slowly becoming just a database frontend with authorization. I think it’s very wrong if not dangerous.

Server will never be a dumb database frontend except for trivial CRUD applications. If you have more than one user in the system working with the same thing concurrently, you have to have some coordination on server side. If you have any business logic at all, you have to have it on the server side.

Client is becoming thicker, fine. It may have a database, offline state and a lot logic. But it does not mean that server is losing any of its thickness in domain model. We’re only moving the presentation side (all of it), enhancing it, maybe duplicating the domain logic. But ultimately we cannot trust the client about anything. It’s all in hands of the users, exposed to reverse engineering, all kinds of forgery and so on. I can send whatever I want to the server with curl.

Next point: With REST you can’t have security. I’m young, green and new to the game, but is that really true? Does REST mean no authorization, no results filtered for each user? If Jim can see some “accounts” in the system and Jessica some others, does it make it impossible to return different results for GET /account depending on context?

To take it a step further: We’re not talking about the very monolithic view here, are we? REST doesn’t mean that you only can have one representation of an “account”, and only one GET/POST/PUT on it. You can still have more representations, each tailored for a specific use case. Getting a list of Jimmy’s friends would be a completely different endpoint from managing user permissions, account settings etc. Even if they’re all “accounts”.

Another point that Chris is making is that for some reason having one resource for one thing is bad, but there’s little explanation for it except for a restaurant counters analogy. It’s hard to argue if there are no real arguments on the other side, but…

Having concrete, well-defined resources that represent one thing each doesn’t really sound like a bad idea to me. If anything, it’s more like the ideal interface segregation and single responsibility. Are they not desirable anymore? Going back to blurry restaurant analogies – I may go for fries or rice, I do order dessert from a different part of the menu than the main course, I may even order wine from a different menu altogether. I may be served by more than one waiter. Heck, sometimes I may even go for a buffet!

To sum up, REST is more than dumb CRUD, and server is much more than just an authorizing database front end. Popularity of REST is a bit of fashion, it’s just another way to solve the problem, and by no means is it the silver bullet. But still, the future may just as well be RESTful.