Marrying ClojureScript and JS Frameworks – Knockout Edition

A while ago I began to play with ClojureScript and tried to get it to work with popular frameworks. I played with a few of them, most recently with Knockout.js. This post sums up those efforts and my not-so-optimistic view on ClojureScript.

  • I tried “bare” jQuery. It was pretty smooth.
  • I tried Backbone.js. I got it to work on a simple example, though one reader on Twitter rightfully commented that ClojureScript was hideous. Yes, that Backbone example is hideous. Later on I tried to do something less trivial. Eventually I fled in horror, thanks to impedance mismatch between heavily OO Backbone and non-OO ClojureScript sauced with my ignorance in CLJS (and Backbone).
  • I also gave Angular.js a shot. It started really smooth, because Angular proudly states that it doesn’t rely on object-oriented programming so much. It was great. Right to the moment when I started arguing with the compiler renaming my variables, soon followed by discovery that Angular and Closure are no go.

So, the time has come to another experiment – this time Knockout.js. I followed the official tutorial and here is what I eventually came up with.

The Page

The complete page in Hiccup looks like this. Nothing particularly exciting here.

(defn render-body []
  (hp/html5
     [:head]
     [:body

      [:p "First name: " [:strong {:data-bind "text: firstName"} "todo"]]
      [:p "Last name: " [:strong {:data-bind "text: lastName"} "todo"]]
      [:p "Full name: " [:strong {:data-bind "text: fullName"} "todo"]]

      [:p "First name: " [:input {:data-bind "value: firstName"}]]
      [:p "Last name: " [:input {:data-bind "value: lastName"}]]

      [:button {:data-bind "click: capitalizeLastName"} "Go caps"]

      (hp/include-js
        "//ajax.aspnetcdn.com/ajax/knockout/knockout-2.1.0.js"
        "js/cljs.js")
      (hp/include-css "css/todo.css")
      ]))

ClojureScript

The most interesting part is the ClojureScript code. Here’s one way to do it:

(ns hello-clojurescript)

(defn app-view-model []
  (this-as this
           (set! (.-firstName this) (.observable js/ko "Bert"))
           (set! (.-lastName this) (.observable js/ko "Bertington"))
           (set!
             (.-fullName this)
             (.computed js/ko
               (fn []
                 (str (.firstName this) " " (.lastName this))) this))
           (set!
             (.-capitalizeLastName this)
             (fn []
               (.lastName this (-> this .lastName .toUpperCase)))))
  nil
  )

(.applyBindings js/ko (app-view-model.))

Yes, I do need to explicitly return “nil” there. Otherwise it returns this.fullName = ..., and that breaks KO.

It works, but it’s hard to defend it in comparison to the JS equivalent:

function AppViewModel() {
    this.firstName = ko.observable("Bert");
    this.lastName = ko.observable("Bertington");
    this.fullName = ko.computed(function() {
        return this.firstName() + " " + this.lastName();
    }, this);
    this.capitalizeLastName = function() {
        var currentVal = this.lastName();
        this.lastName(currentVal.toUpperCase());
    };

}

ko.applyBindings(new AppViewModel());

Complete code can be found at my GitHub repository.

Better Way – Macro

This code can be made a lot better with a custom macro, as the one presented at StackOverflow:

(defvar name_model
    first_name (observable "My")
    last_name (observable "Name")
    name (computed (fn [] (str (. this first_name) " " (. this last_name)))))

(. js/ko (applyBindings name_model));

Now, that would be something!

… except for that defining macros in ClojureScript is harder than in plain Clojure, to the point that I haven’t gotten it to work yet.

Conclusions on ClojureScript

I spent quite a few hours poking at ClojureScript, and I have mixed feelings.

  1. It’s pretty hard to get ClojureScript to work with existing JS frameworks, mostly because using objects in CLJS requires so much ceremony.
  2. Contrary to plain Clojure, “fun” and “productive” aren’t the words that come to mind when I think of my adventures in CLJS. “Frustrating” and “intimidating” are much more appropriate. I’m constantly arguing with the compiler and trying to beat it to do the right thing, not having fun solving problems.
  3. Some stuff can be covered with macros, but all in all it feels very… rigid and constraining. I feel like every once in a while I’m bound to hit another rough corner, spend too much time on it, write another macro, and so on. All that only to bridge the gaps and make ClojureScript look more like… JavaScript. In fact, that feels like writing my own layer of macros to compile JS-like-DSL to ClojureScript.
  4. Perhaps there is a better way and I’m just doing something wrong. Maybe you’re not supposed to use those frameworks at all, but roll your own or use those few written in ClojureScript?
  5. Perhaps all of this makes little sense on such a small scale, and you need something really big to appreciate ClojureScript. You just need to invest many hours in passing the learning curve and macroing your way out. Maybe then it becomes more productive, modular and whatnot. I don’t know, in fact I have too little experience in JavaScript itself to answer such questions. I’m not very optimistic about it, though.

I am a DZone Most Valuable Blogger and JavaCodeGeek!

I’m happy to announce that at about the same time I became a DZone Most Valuable Blogger and a JavaCodeGeeks author. Some of my posts have been appearing on various blogs, community sites and aggregators for a while (especially those on Clojure and functional programming), but being invited to participate in something like these two programs is quite a new experience.

Some parts of this blog is ranting and bragging, but the main purpose has always been to learn and share knowledge. I wouldn’t have learned so much without the blog – just getting something to work is a lot easier than understanding it enough to be able to explain it to someone. We all know that the community is quite demanding. I learned the hard way that writing too shallowly, repetitively, or without proper understanding is immediately punished. On the other hand, it’s truly exhilarating to see my post mentioned by people I really admire, or make it to the front page of Hacker News.

The most rewarding part is always the comments. Not only does it mean that someone has visited the site, but also that they actually cared to read the post and share their insights or related knowledge.

Thanks, guys!

Version-Based Optimistic Concurrency Control in JPA/Hibernate

This article is an introduction to version-based optimistic concurrency control in Hibernate and JPA. The concept is fairly old and much has been written on it, but anyway I have seen it reinvented, misunderstood and misused. I’m writing it just to spread knowledge and hopefully spark interest in the subject of concurrency control and locking.

Use Cases

Let’s say we have a system used by multiple users, where each entity can be modified by more than one user. We want to prevent situations where two persons load some information, make some decision based on what they see, and update the state at the same time. We don’t want to lose changes made by the user who first clicked “save” by overwriting them in the following transaction.

It can also happen in server environment – multiple transactions can modify a shared entity, and we want to prevent scenarios like this:

  1. Transaction 1 loads data
  2. Transaction 2 updates that data and commits
  3. Using state loaded in step 1 (which is no longer current), transaction 1 performs some calculations and update the state

In some ways it’s comparable to non-repeatable reads.

Solution: Versioning

Hibernate and JPA implement the concept of version-based concurrency control for this reason. Here’s how it works.

You can mark a simple property with @Version or <version> (numeric or timestamp). It’s going to be a special column in database. Our mapping can look like:

@Entity
@Table(name = "orders")
public class Order {
	@Id
	private long id;

	@Version
	private int version;

	private String description;

	private String status;

	// ... mutators
}

When such an entity is persisted, the version property is set to a starting value.

Whenever it’s updated, Hibernate executes query like:

update orders
set description=?, status=?, version=?
where id=? and version=?

Note that in the last line, the WHERE clause now includes version. This value is always set to the “old” value, so that it only will update a row if it has the expected version.

Let’s say two users load an order at version 1 and take a while looking at it in the GUI.

Anne decides to approve the order and executes such action. Status is updated in database, everything works as expected. Versions passed to update statement look like:

update orders
set description=?, status=?, version=2
where id=? and version=1

As you can see, while persisting that update the persistence layer increments the version counter to 2.

In her GUI, Betty still has the old version (number 1). When she decides to perform an update on the order, the statement looks like:

update orders
set description=?, status=?, version=2
where id=? and version=1

At this point, after Anne’s update, the row’s version in database is 2. So this second update affects 0 rows (nothing matches the WHERE clause). Hibernate detects that and an org.hibernate.StaleObjectStateException (wrapped in a javax.persistence.OptimisticLockException).

As a result, the second user cannot perform any updates unless he refreshes the view. For proper user experience we need some clean exception handling, but I’ll leave that out.

Configuration

There is little to customize here. The @Version property can be a number or a timestamp. Number is artificial, but typically occupies fewer bytes in memory and database. Timestamp is larger, but it always is updated to “current timestamp”, so you can actually use it to determine when the entity was updated.

Why?

So why would we use it?

  • It provides a convenient and automated way to maintain consistency in scenarios like those described above. It means that each action can only be performed once, and it guarantees that the user or server process saw up-to-date state while making a business decision.
  • It takes very little work to set up.
  • Thanks to its optimistic nature, it’s fast. There is no locking anywhere, only one more field added to the same queries.
  • In a way it guarantees repeatable reads even with read committed transaction isolation level. It would end with an exception, but at least it’s not possible to create inconsistent state.
  • It works well with very long conversations, including those that span multiple transactions.
  • It’s perfectly consistent in all possible scenarios and race conditions on ACID databases. The updates must be sequential, an update involves a row lock and the “second” one will always affect 0 rows and fail.

Demo

To demonstrate this, I created a very simple web application. It wires together Spring and Hibernate (behind JPA API), but it would work in other settings as well: Pure Hibernate (no JPA), JPA with different implementation, non-webapp, non-Spring etc.

The application keeps one Order with schema similar to above and shows it in a web form where you can update description and status. To experiment with concurrency control, open the page in two tabs, do different modifications and save. Try the same thing without @Version.

It uses an embedded database, so it needs minimal setup (only a web container) and only takes a restart to start with a fresh database.

It’s pretty simplistic – accesses EntityManager in a @Transactional @Controller and backs the form directly with JPA-mapped entity. May not be the best way to do things for less trivial projects, but at least it gathers all code in one place and is very easy to grasp.

Full source code as Eclipse project can be found at my GitHub repository.

JDD 2012

Last week I attended the JDD 2012 conference. Here’s a bunch of very quick notes from the talks.

  • Rebecca Wirfs Brock, Joseph Yoder “Pragmatic, Not Dogmatic TDD: Rethinking How We Test” – nice presentation about TDD, felt a bit like theater (dialogue between two presenters). A few good points to show that TDD is not only about the fast red-green cycle (anyone thinks so, by the way?). Little gem: Use pomodoro specifically to break the flow, take a step back and look at the big picture.
  • Jarosław Pałka “The deconstruction of architecture in times of crisis” – very good talk about applying system thinking to software development. The theory is fairly well applicable to our craft, and the interdisciplinary view helps you see it in different perspective. Briefly explained some pieces of the theory as well as a few concrete examples (tragedy of the commons, boiling frog syndrome). I only found one important bit missing: Theory of constraints. Suggested reading: Thinking in Systems: A Primer.
  • Jakub Kubryński “Microbenchmarks performance in the smaller scale” – difficult choice (in conflict with Jessica Kerr’s FP/OOP talk), but I went for this one since the topic was less familiar. And I don’t regret it. I learned a ton about Java execution model: The role and place of JIT, possible optimizations, and finally the impact on microbenchmarks (and ways to do them better).
  • Sławomir Sobótka “Evolutionary Distillation of Architecture” (in Polish, I took the liberty of translating the title) – introduction to ports & adapters (aka hexagonal architecture). As usually, served in a broader context on thinking about how we code, what tools we use and why we should constantly learn new ones. As always, very entertaining, professional and inspiring. Little gem: If someone only knows entity and service, he’ll go at great lengths implementing everything with just those two concepts. Related reading: Hexagonal Architecture by Alistair Cockburn.
  • Piotr Bucki on cross site scripting attacks – nice presentation on XSS, discussing various ways to protect an application: input sanitization and validation, escaping on input and output, white list filtering for places which need to allow special content (like auction descriptions with limited HTML). Conclusion: Sanitize and validate input, escape everything on output. Careful, include all directions and outputs – validate batch import from files, don’t even trust your own logs (someone could deliberately execute an invalid request to have it logged, and then XSS into your admin log viewer). Solid, though not spectacular talk.
  • Wiktor Żołnowski “Reversed Tests Pyramid – dealing with legacy code” – great idea on approach to testing legacy code that can’t easily be covered with unit tests. Start with end-to-end tests (in bigger number that would otherwise be necessary) to gain some understanding and confidence in refactoring. Write more and more lower level tests as you refactor. Eventually throw away slow higher level tests as they become redundant and invert the pyramid as it should be. Quite fun, felt almost like a BOF session (appropriate for the evening). Hated one bit: Looking down upon the audience and repeating “I moved from development to coaching a few years ago”, “This is probably not interesting to you, developers” etc.
  • Thomas Sundberg “How to fail a software project fast and efficiently?” – pretty bad presentation. Could’ve been fun and sarcastic, but was really shallow and dull. Very little content, no argumentation.
  • Adam Bien “Java EE Future Is Now, But It Is Not Evenly Distributed Yet” – positive suprise! Went there just because I know so little about recent JEE and wanted to see something new. Very professional and dense. On our eyes Adam built a JEE app from scratch that demonstrated dependency injection (including configuration/params), differences between EJB and non-EJB life cycles, JSF, asynchronous execution (comet), event propagation… Very good mix of humor and live coding. Repeat, that was one spectacular live coding. I expected some fanaticism, but got none of that sort – some respect to Spring and others, but actually focusing on showing how JEE became lean and cool, not bashing the competition.
  • Patrycja Wegrzynowicz “Security Vulnerabilities in OpenSource Java Libraries” – stats done bad. Most of the presentation was a view statistical views on data for number of vulnerabilities discovered in various libraries and app servers. Ignoring information on how quickly the holes are patched (or whether there are workarounds known to community), it literally contained information such as over X years 100 issues have been discovered in Tomcat, 20 in JBoss, so Tomcat is much more likely to be insecure. But app servers from IBM and Oracle are even worse, they’ve had more issues discovered over so many years.
  • Hardy Ferentschik “JPA into the Cloud Introducing OGM and OpenShift” – demonstrated an experimental Hibernate extension to persist JPA to NoSQL databases (and deploy that to cloud using OpenShift). Not yet production ready, has some rough edges and I’m not completely sure why you’d want to do that. Anyway – challenging, interesting, well prepared and presented.
  • Gustaf Nilsson Kotte “Surviving the Zombie Apocalypse of Connected Devices” – proposal to use HTML hypermedia as the data format for all kinds of environments. Create a basic HTML model that can be viewed by any browser (leverage the fact that browser can display HTML data in a nice way). That can be used directly as default for most non-critical screens. For more interesting (or critical) areas or environments, use HTML merely as data format and build an app on top of that. We have tools to parse and generate HTML, so why not use it as data format? Very interesting idea – instead of REST and all kinds of unprintable data formats, use a popular format that has decent support and can be displayed by any device. The talk was pretty hard to follow, though – too calm and sleepy.
  • Jessica Kerr “Git Happens” – very good talk on Git to close the conference. No live demos, very good explanation of basic concepts. Not just the happy path, and not overly optimistic. Also, Jessica is an excellent speaker – passionate (sometimes to the point of running, screaming and damaging equipment), but also with a very good plan and attention to detail.

All in all, I’m quite satisfied with this conference. Like any, it had some low points and on one or two occasions I skipped the talks because there was nothing interesting to me in that hour. Feels small and local compared to 33rd Degree, and doesn’t have the community vibe of Confitura, but certainly has its place. Anyway, I learned a few new things, most of the presentations were good and a few were great, so it definitely was worth it.

Hello, Backbone and ClojureScript

A few days ago I started learning ClojureScript. I wrote a trivial “hello world” application just to get ClojureScript to compile and execute, and later added some basic jQuery support with jayq.

The time has come to make things a little bit more interesting and add Backbone.js to the mix. I’ve never done ClojureScript or Backbone before, so I’m learning them at the same time with an interesting learning curve.

Anyway, I managed to rewrite the first two examples from Backbone docs to pure CLJS. I made some minor modifications like triggering events on button click and changing main background instead of sidebar.

Here’s my page source (with Hiccup):

(hp/html5
  [:head]
  [:body
   [:button#clickable-event "Click to trigger an alert from basic Backbone event"]
   [:button#clickable-color "Click to change background color"]
   (hp/include-js
     "http://code.jquery.com/jquery-1.8.2.min.js"
     "http://underscorejs.org/underscore.js"
     "http://backbonejs.org/backbone.js"
     "js/cljs.js")
   ])

As you can see, it renders a very basic page with two buttons and includes a few JS libraries.

And here’s the CLJS file mixing jQuery and Backbone:

(ns hello-clojurescript
  (:use [jayq.core :only [$]])
  (:require [jayq.core :as jq]))

; ALERT ON CLICK
; Rewrite of http://backbonejs.org/#Events
(def o {})

(.extend js/_ o Backbone.Events)

(.on o "alert"
  (fn [msg] (js/alert msg)))

(jq/bind ($ "#clickable-event") :click
      (fn [e] (.trigger o "alert" "Hello Backbone!")))

; MODEL WITH COLOR CHOOSER
; Inspired by http://backbonejs.org/#Model but without sidebar

(def MyModel
  (.extend Backbone.Model
    (js-obj
      "promptColor"
      (fn []
        (let [ css-color (js/prompt "Please enter a CSS color:")]
          (this-as this
                   (.set this (js-obj "color" css-color))))))))

(def my-model (MyModel.))

(.on my-model "change:color"
  (fn [model color]
    (jq/css ($ "body") {:background color})))

(jq/bind ($ "#clickable-color") :click
         (fn [e] (.promptColor my-model)))

There’s a number of new things (to me) and nonobvious pitfalls. View this side-by-side with Backbone demos, and note:

  • To invoke _.extend(o, Backbone.Events), do (.extend js/_ o Backbone.Events). ClojureScript will correctly transform (.extend js/_ ...) to _.extend(...), and it will copy Backbone.Events as is (no quoting necessary)
  • To distinguish between objects and functions defined elsewhere and in CLJS, always prefix the former with js/name. Works for alert, underscore etc.
  • I had an issue with passing objects (as maps) directly to calls like Backbone.Model.extend(). Tried things like {:promptColor fn} and {"promptColor" fn} to no avail. I finally discovered (js-obj) and it did the trick, but it’s pretty cumbersome. I wonder if there’s a better way.
  • You need some extra work to use this. It has to be bound to a Clojure symbol with this-as macro.
  • On a slightly related note, I really begin to love jayq. In this example I use bare Backbone directly and struggle, and really appreciate jayq bridging the gap to jQuery. I wonder if there is a CLJS wrapper for Backbone.

All in all, it’s an interesting exercise. Just the right learning curve – stimulating, but not discouraging, regularly providing visible feedback.

As usually, complete source is at GitHub. I created a new repository for it, to keep “hello ClojureScript” as small as possible. This new demo probably will grow as I learn more Backbone.

Hello, ClojureScript! (with jQuery)

I decided to give ClojureScript a try. It did not come easy, because I found the official documentation somewhat complicated. I know there is ClojureScript One, but that project also is not as simple as it could be. I don’t want fancy functionality, noir/compojure, enlive/hiccup, and tons of other semi-relevant tools. Bare simplistic HTML and a starting hook for ClojureScript is pretty much all I need for the head start, I can add the rest later.

I was looking for something really minimal, and the first simple example on my Google search was Daniel Harper’s article. I got rid of noir, used up to date versions of libraries, and voila – it’s working!

When I had my first “hello world” alert showing on page load, I decided to make things a little bit more interesting and introduce jQuery. I found jayq from Chris Granger and decided to give it a shot. There’s also a sample app on Chris’ blog that helped me with some issues, namely figuring out how to bind events. It references a few more interesting libs (namely fetch & crate), but I’ve had enough for now. I guess I could spend the whole night chasing such references.

In the end, the interesting pieces of code are below:

project.clj (configured to compile CLJS from src-cljs to resources/public/js/cljs.js):

(defproject hello-clojurescript "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]
                 [ring "1.1.6"]
                 [jayq "0.1.0-alpha3"]]
  :plugins [[lein-cljsbuild "0.2.8"]]
  :cljsbuild
  {
   :source-path "src-cljs"
   :compiler
   {
    :output-to "resources/public/js/cljs.js"
    :optimizations :simple
    :pretty-print true
    }
   }
  :main hello-clojurescript.core
  )

core.clj (trivial app, with Ring wrapper configured to serve JS resources):

(ns hello-clojurescript.core
  (:require [ring.adapter.jetty :as jetty]
            [ring.middleware.resource :as resources]))

(defn handler [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body
   (str "<!DOCTYPE html>"
        "<html>"
        "<head>"
        "</head>"
        "<body>"
        "<p id=\"clickable\">Click me!</p>"
        "<p id=\"toggle\">Toggle Visible</p>"
        "<script src=\"http://code.jquery.com/jquery-1.8.2.min.js\"></script>"
        "<script src=\"js/cljs.js\"></script>"
        "</body>"
        "</html>")})

(def app
  (-> handler
    (resources/wrap-resource "public")))

(defn -main [& args]
  (jetty/run-jetty app {:port 3000}))

hello-clojurescript.cljs (this one gets compiled to JavaScript):

(ns hello-clojurescript
  (:use [jayq.core :only [$ delegate toggle]]))

(def $body ($ :body))

(delegate $body :#clickable :click
          (fn [e]
            (toggle ($ :#toggle))))

Complete source code with instructions can be found at my GitHub repository.

At the moment I see the following issues:

  • I’m really green at ClojureScript. Tons to learn here!
  • The JavaScript file compiled from this trivial example is 13k lines long and weighs about 500 kb. Doh! Fine for local development on desktop, not that good for targetting mobile.
  • The official docs for ClojureScript are really… discouraging. Just like core Clojure documentation, they are pretty academic and obscure.
  • Docs for jayq are… Wait a minute, nonexistent? At least it’s a fairly thin adapter with small, comprehensible codebase.

DevDay 2012

On October 5, 2012 I attended the second edition of DevDay, a one-day conference sponsored and organized by ABB in Krakow.

The Awesome

Scott Hanselman was the first speaker, and also one of the main reasons why I went to the conference. Even though I only knew him from This Developer’s Life.

His “Scaling Yourself” talk was probably the best productivity talk I’ve seen so far. Part of it was tricks and techniques I have already known, such as GTD and pomodoro. Apart from that, my notes include:

  • The fewer things you do, the more of each you can do.
  • By doing something you are likely to get more of it. If you’re available on weekends and after hours, you will be expected to do it. If you’re available for calls about work on vacations, you will be called. Even if you do work at that time, set that email to go out at 9 AM next business morning.
  • Avoid “guilt systems” such as long collection of recorded TV shows – or books, or unread articles. Or whatever it is that you collect and want to do, but eventually forms a big pile that you only feel guilty about.
  • Sort your information streams by importance and limit usage. Some obvious ones like do less Twitter or Facebook. Some less obvious ones like basic email filters. Combine that with techniques that help you root out the distractions such as pomodoro or Rescue Time.
  • Use information aggregators: Blogs or sites that repost articles, mashups etc. instead of subscribing to 100 different blogs and news sites.
  • Do not multitask, except for things that go well together. Foe example, exercise while watching TV or listening to podcasts. What’s more: use activities you want to do to motivate things you should do. For instance, watch TV only as long as you keep on moving on the treadmill.
  • Plan your work: Find three things you want to do today, this week and this year, and do them. Helps you set the focus on goals. Hint: Email & twitter probably won’t be on the list.
  • Plan your work: Plan your own work sprints, execute, and finally perform retrospectives. Can be applied to work, but also all kinds of personal activities.
  • Synchronize to paper. Don’t limit your space to one screen when you can print or write/draw it on as many sheets of paper as you want. Also, paper notebooks can be used in different conditions and never run out of battery..

Finally, Scott is a great speaker. Lots and lots of content served in a perfect way, sauced with some pretty good jokes. Delicious.

That’s just a bunch of quick notes. Even if you think you’ve heard enough on the topic of productivity, go watch the presentation on Scott’s site. Now.

The Good

I enjoyed the “Why You Should Talk to Strangers” talk from Martin Mazur. Half of it was about social interactions: “us versus them” divisions, difficulties that one can face when approaching complete strangers in public, and the amount of new things you can learn from them if you break the ice. The rest was really about polyglot programming: learning Eiffel, Haskell or Ruby and applying some concepts and ideas back to C#. Largely an unknown territory, but still the talk resonated really well with me.

Antek Piechnik gave a good talk on continuous delivery at an extreme scale: Where each new team member pushes code to production during the first week, and each commit to master can trigger deployment. I found the ideas on project organization pretty controversial, though: having a totally flat team, large pool of features and everyone working on what they want and how they want. Sounds interesting, but it’s only possible if you have only experienced A++ programmers in team that also have the same vision, agree on tools and techniques, and so on.

Finally, I liked Greg Young’s talk on getting productive in a project in 24 hours. The talk was nothing like the subject, though. Basically an introduction to code analysis: afferent/efferent coupling, test coverage and cyclomatic complexity, as well as data mining VCS. Very clear and down-to-earth, discussing some tools and practical examples for each concept.

I particularly liked the points on code coverage. I knew that method with 20 possible paths can have 100% line coverage from 2 tests, but still be poorly tested. Greg made a good point explaining that method coverage gets worse as the number of methods/collaborators between the test and the method grows, or when the methods between the test and our method have high cyclomatic complexity. In such cases it’s really an accidental coverage that in no way guarantees security.

Greg repeatedly stressed how all these concepts are only tools. They indicate interesting areas that may be trouble spots and may need attention. But one can never say that cyclomatic complexity of X is good code and Y is bad code (and the same is true for all other metrics).

The Rest

Rob Ashton’s talk on JavaScript was alright, but not spectacular. I did not learn much new. I know that the language is here to stay, for better or worse. You can patch some gaps with jslint/jshint and others with CoffeeScript, but for its spread the tooling is really patchy and barely existent.

I skipped Mark Rendle’s talk on Simple.Data/Simple.Web. Not my area.

I really did not like Sebastien Lambla’s talk on HTTP caching. Noise to signal ratio approaching infinity. Filled with poor jokes and irrelevant comments. Little, chaotically and patchily discussed substance.

Wrapping Up

All in all, DevDay was a good conference. Really good selection of speakers on a free conference, with free lunch, coffee and snacks. No recruiters, no stands, just the participants and speakers. I wish it had at least two tracks and more focus. I’m not sure if it’s a generic developer conference or a .net event (went for the former, felt atmosphere of the latter).

Configuration Files in Clojure

I recently made a contribution to ghijira, a small tool written in Clojure for exporting issues from GitHub in JIRA-compatible format. One of the problems to solve there was loading configuration from file.

Originally, it used have a separate config.clj file that looked like this:

(def auth "EMAIL:PASS")
(def ghuser "user-name")
(def ghproject "project-name")
(def user-map
  { "GithubUser1" "JIRAUser1"
    "GithubUser2" "JIRAUser2"})

Then it was imported in place with:

(load-file "config.clj")

I did not like it, because it did not feel very functional and the configuration file had too much noise (isn’t def form manipulation too much for a configuration file?).

For a moment I thought about using standard Java .properties files. They get the job done, but they’re also somewhat rigid and unwieldy.

It occurred to me I really could use something similar to Leiningen and its project.clj files: Make the config a plain Clojure map. It’s very flexible with minimal syntax, and it’s pure data just like it should be.

From a quick Google search I found the answer at StackOverflow.

It turns out I can rewrite my configuration file to:

{:auth      "EMAIL:PASS"
 :ghuser    "user-name"
 :ghproject "project-name"
 :user-map
   { "GithubUser1" "JIRAUser1"
     "GithubUser2" "JIRAUser2" }
}

And then load it in Clojure with read:

(defn load-config [filename]
  (with-open [r (io/reader filename)]
    (read (java.io.PushbackReader. r))))

That’s it, just call read.

I find this solution a lot more elegant and “pure”.

By the way, this data format is getting a life of its own, called Extensive Data Notation. The Relevance team has even published a library to use EDN in Ruby.

Careful with def in Clojure

Let’s start with a puzzle. Let’s create a little Leiningen project called careful. Let’s set :main careful.core in project.clj and put this in careful/core.clj:

(ns careful.core
  (:gen-class))

(defn get-my-value []
  (println "Sleeping...")
  (Thread/sleep 5000)
  (println "Woke up")
  "Done")

(def my-def (get-my-value))

(defn -main [& args]
  (println "Hello, World!"))

Here’s the question: What happens when you compile this project?

And the answer is…

$ time lein2 compile
Compiling careful.core
Sleeping...
Woke up
Compilation succeeded.

real	0m7.098s
user	0m5.276s
sys	0m0.212s

Hey, my program actually printed something during compilation! And it took way too much time. I didn’t expect that.

Such an apparently innocuous def can get you in a lot of trouble. Sure, no-one puts a sleep like that, but what about:

  • Code that computes something, perhaps something taking time or space?
  • Code that loads something from the network?

I don’t yet understand why it is resolved at compile time.

But I can understand why using def in that way is not a good idea. It’s an old imperative habit. This may be a perfectly valid imperative program:

public static void main(String[] args) {
	Data data = loadDataFromInternet();
	ProcessedData proc = process(data);
	generateReport(proc);
}

You may be tempted to do it this way in Clojure:

(def data (load-data-from-internet))
(def proc (process data))
(defn -main [& args]
  (generate-report proc))

… but that’s still an imperative style and it feels wrong.

It also is real pain to test.

How about one of these equivalents?

(defn -main [& args]
  (let [data (load-data-from-internet)]
     (generate-report (process proc))))
(defn -main [& args]
   (generate-report (process (load-data-from-internet))))

In the end, I arrived at the following conclusion. You should only use def for constants, some global parameters, dynamic variables, definitions of higher order functions – that kind of static stuff. All logic and behavior belongs in functions.

Update

As djork pointed out at Reddit, it’s because def creates a var in the current namespace with specific value.

It makes some sense when you think of what defn looks like – it’s really a macro wrapping def (also pointed out by djork). And we do expect functions introduced by defn to be compiled, right. Even docs clearly state that defn is the “same as (def name (fn [params* ] exprs*))“.

I still find it very confusing, though. I wonder if I’m just abusing the language.

Second Update

I came back to it later, and I may have finally understood.

This is a perfectly valid statement:

(def my-def (get-my-value))

But what about these?

; Unexpected argument:
(def my-def (get-my-value))

; Type cast exception (vector to number)
(def my-def-2 (+ 2 []))

; Whatever invalid statement
(def my-def-2 (+ 2 +))

Should they throw a compile-time error? It makes sense, right?

Now, when you type this:

(def my-def (get-current-date))

At runtime, do you expect it to have state as of compile time, or as of run time? In other words, should it be date of compilation, or “now” at the time of execution? The latter, right?

I can see why both evaluations (at compile and run time) are needed. Depending on point of view, it’s either some sort of language fragility or developer abusing the language. Either way, the conclusion stays the same: Careful with that def, Eugene.

Discussion

Aside from this blog, there is an interesting discussion with more detail at Reddit. Thanks guys!

Domain Modeling: Naive OO Hurts

I’ve read a post recently on two ways to model data of business domain. My memory is telling me it was Ayende Rahien, but I can’t find it on his blog.

One way is full-blown object-relational mapping. Entities reference each other directly, and the O/R mapper automatically loads data for you as you traverse the object graph. To obtain Product for an OrderLine, you just call line.getProduct() and are good to go. Convenient and deceptively transparent, but can easily hurt performance if you aren’t careful enough.

The other way is what that post may have called a document-oriented mapping. Each entity has its ID and its own data. It may have some nested entities if it’s an aggregate root (in domain-driven design terminology). In this case, OrderLine only has productId, and if you want to get the product you have to call ProductRepository.getProduct(line.getProductId()). It’s a bit less convenient and requires more ceremony, but thanks to its explicitness it also is much easier to optimize or avoid performance pitfalls.

So much for the beforementioned post. I recently had an opportunity to reflect more on this matter on a real world example.

The Case

The light dawned when I set out to create a side project for a fairly large system that has some 200+ Hibernate mappings and about 300 tables. I knew I only needed some 5 core tables, but for the sake of consistency and avoiding duplication I wanted to reuse mappings from the big system.

I knew there could be more dependencies on things I don’t need, and I did not have a tool to generate a dependency graph. I just included the first mapping, watched Hibernate errors for unmapped entities, added mappings, checked error log again… And so on, until Hibernate was happy to know all the referenced classes.

When I finished, the absolutely minimal and necessary “core” in my side project had 110 mappings.

As I was adding them, I saw that most of them are pretty far from the core and from my needs. They corresponded to little subsystems somewhere on the rim.

It felt like running a strong magnet over a messy workplace full of all kinds of metal things when all I needed was two nails.

Pain Points

It turns out that such object orientation is more pain than good. Having unnecessary dependencies in a spin-off reusing the core is just one pain point, but there are more.

It also is making my side project slower and using too many resources – I have to map 100+ entities and have them supported in my 2nd level cache. When I’m loading some of the core entities, I also pull many things I don’t need: numerous fields used in narrow contexts, even entire eagerly-loaded entities. At all times I have too much data floating around.

Such a model also is making development much slower. Build and tests take longer, because there are many more tables to generate, mappings to scan etc.

It’s also slower for another reason: If a domain class references 20 other classes, how does a developer know which are important and which are not? In any case it may lead to very long and somewhat unpleasant classes. What should be core becomes a gigantic black hole sucking in the entire universe. When an unaware newbie goes near, most of the time he will either sink trying to understand everything, or simply break something – unaware of all the links in his context, unable to understand all links present in the class. Actually, even seniors can be deceived to make such mistakes.

The list is probably much longer.

Solution?

There are two issues here.

How did that happen?

I’m writing a piece of code that’s pretty distant from the core, but could really use those two new attributes on this core entity. What is the fastest way? Obvious: Add two new fields to the entity. Done.

I need to add a bunch of new entities for a new use case that are strongly related to a core entity. The shortest path? Easy, just reference a few entites from the core. When I need those new objects and I already have the old core entity, Hibernate will do the job of loading the new entities for me as I call the getters. Done.

Sounds natural and I can see how I could make such mistakes a few years ago, but the trend could have been stopped or even reversed. With proper code reviews and retrospectives, the team may have found a better way earlier. Having some slack and good will it may have even refactored the existing code.

Is there a better way to do it?

Let’s go back to the opening section on two ways to map domain classes: “Full-blown ORM” vs. document/aggregate style.

Today I believe full-blown ORM may be a good thing for a fairly small project with a few closely related use cases. As soon as we branch out new bigger chunks of functionality and introduce more objects, they should become their own aggregates. They should never be referenced from the core, even though they themselves may orbit around and have a direct link to the core. The same is true for the attributes of core entites: If something is needed in a faraway use case, don’t spoil the core mapping with a new field. Even introduce a new entity if necessary.

In other words, learn from domain-driven design. If you haven’t read the book by Eric Evans yet, go do it now. It’s likely the most worthwhile and influential software book I’ve read to date.