Monthly Archives: April 2012

Software for Use

Life, Programming,

Here’s confession of a full time software developer: I hate most software. With passion.

Why I Hate Software

Software developers and people around the process are often very self-centered and care more about having a good time than designing a useful product. They add a ton of cool but useless and bugged features. They create their own layers of frameworks and reinvent everything every time, because writing code is so much more fun than writing, reusing or improving it.

They don’t care about edge cases, bugs, rare conditions and so on. They don’t care about performance. They don’t care about usability. They don’t care about anything but themselves.

Examples? Firefox that has to be killed with task manager because it slows to a crawl during the day on most powerful hardware. Linux which never really cared or managed to solve the issues with drivers for end user hardware. Google maps showing me tons of hotel and restaurant names instead of street names, the exact opposite of what I want when planning a trip. Eclipse or its plugins that require me to kill the IDE from task manager, waste some more time, and eventually wipe out the entire workspace, recreate it and reconfigure.

All the applications with tons of forms, popups, dialogs and whatnot. Every error message that is a page long, has a stacktrace, cryptic code and whatever internal stuff in it. All the bugs and issues in open source software, which is made in free time for fun, rarely addressing edge cases or issues happening to a few percent users because they’re not fun.

It’s common among developers to hate and misunderstand the user. It’s common even at helpdesk, support and many people who actually deal with end users. In Polish there is this wordplay “użyszkodnik”, a marriage of “użytkownik” (user) and “szkodnik” (pest).

What Software Really Is About

Let me tell you a secret.

The only purpose of software is to serve. We don’t live in a vacuum, but are always paid by someone who has a problem to solve. We are only paid for two reasons: To save someone money, or to let them earn more money. All the stakeholders and users care about it is solving their problems.

I’ve spent quite a few years on one fairly large project that is critical for most operations of a corporation. They have a few thousand field workers and a few dozen managers above, and only a handful of people responsible for software powering all this. Important as it is, the development team is a tiny part of the entire company.

Whenever I design a form, a report, an email or whatever that the end user will ever see, the first and most important thing to do is: Get in their shoes. Understand what they really need and what problem they are trying to solve. See how we can provide it to the them so that it’s as simple, concise, self-explanatory and usable as possible. Only then we can start thinking about code and the entire backend, and even then the most important thing to keep in mind is the end user.

We’re not writing software for ourselves. Most of the time we’re not writing it for educated and exceptionally intelligent geeks either. We write it for housewives, grandmas, unqualified workers, accountants, ladies at bookshops or insurance companies, all kinds of business people.

We write it for people who don’t care about software at all and do not have a thorough understanding of it. Nor do they care care how good a time you were having while creating it. They just want to have the job done.

You’re Doing It Wrong

If someone has to ask or even think about how something works, it’s your failure. If they perform some crazy ritual like rebooting the computer or piece of software, or wipe out a work directory, that’s your fault. If they have to go through five dialogs for a job that could be done with two clicks, or are forced to switch between windows when there is a better way, it’s your failure. When they go fetch some coffee while a report that they run 5 times a day is running, it’s your fault. If there is a sequence of actions or form entries that can blow everything up, a little “don’t touch this” red button, it’s your fault. Not the end user’s.

It’s not uncommon to see a sign in Polish offices that reads (sadly, literally): “Due to introduction of a computer system, our operations are much slower. We are sorry for the inconvenience.” Now, that’s a huge, epic failure.

Better Ways

That’s quite abstract, so let me bring up a few examples.

IKEA. I know furniture does not seem as complicated as software, but it’s not that trivial either. It takes some effort to package a cabinet or a chest of drawers in a cardboard box that can be assembled by the end user. They could deliver you some wood and a picture of cabinet, and blame you for not knowing how to turn one into another. They could deliver a bunch of needlessly complicated parts without a manual, and blame the user again.

They know they need to sell and have returning customers, not just feel good themselves and blame others.

What they do is carefully design every single part and deliver a manual with large, clear pictures and not a single line of text. And it’s completely fool-proof and obvious, so that even such a carpentry ignorant as you can assemble it.

LEGO. Some sets have thousands of pieces and are pretty complex. So complex that it would be extremely difficult even for you, craftsman proficient in building stuff, to reproduce.

Again, they could deliver 5,000 pieces and a single picture to you and put the blame on you for being unable to figure it out. Again, that’s not what what they do. They want to sell and they want you to return. So they deliver a 200-page-long manual full of pictures, so detailed and fool-proof that even a child can do it.

There are good examples in software world as well. StackOverflow is nice, but only for certain kind of users. It’s great for the Internet geeks who get the concept of upvotes, gamification, focusing on tiny narrow parts and not wider discussion etc. Much less for all kinds of scientists and, you know, regular people, who seem to be the intended audience of StackExchange.

Google search and maps (for address search, intuitiveness and performance), DuckDuckGo are pretty good. Wolfram Alpha. Skyscanner and Himpunk. Much of the fool-proof Apple hardware and software.

In other words, when you know what it does and how to use it the first time you see it, and it Just Works, it’s great.

Conclusion

Successful startups know it. They want to sell and if they make people think or overly complicate something, people will just walk on by. I guess many startups fail because they don’t realize it. Many established brands try to do it and learn from startups, simplifying and streamlining their UIs (Amazon, MS Office, Ebay…). It’s high time we applied it to all kinds of software, including the internal corporate stuff and open source.

After all, we’re only here to serve and solve problems of real people.

That’s the way you do it.

“Programming Concurrency on the JVM”

Clojure, Programming, , ,

A few years ago when I took concurrency classes pretty much everything I was told was that in java synchronized is key. That’s the way to go, whenever you have multithreading you have to do it this way, period. I also spent quite a while solving many classic and less classic concurrency problems using only this construct, reimplementing more fancy locks using only this construct, preventing deadlocks, starvation and everything.

Later in my career I learned that is not the only way to go, or at least there are those fancy java.util.concurrent classes that take care of some stuff for you. That was nice, but apparently I never took enough time to actually stop and think how those things work, what they solve and why.

The light dawned when I started reading Programming Concurrency on the JVM: Mastering Synchronization, STM, and Actors by Venkat Subramaniam.

The book starts with a brief introduction on why concurrency is important today with its powers and perils. It quickly moves on to a few examples of different problems: IO-intensive task like calculating size of a large directory, and computationally intensive task of calculating prime numbers. Once the ground is set, it introduces three approaches to concurrent programming.

The first way to do it is what I summed up in the first paragraph, and what Venkat calls the “synchronize and suffer” model. Been there, done that, we know how bad it can get. This approach is called shared mutability, where different threads mutate shared state concurrently. It may be tamed (and a few ways to do it are shown in the book), but is a lot harder than it seems.

Another approach is isolated mutability, where each mutable part of state is only accessed by one thread. Usually this is the actor based concurrency model. The third way is pure immutability where there simply is no mutable state. That is typical for functional programming.

In the following chapters the book explores each of those areas in depth. It briefly explains the Java memory model nad shows what options for dealing with shared mutability and coordinating threads exist in core Java. It clearly states why the features from Java 5 are superior to the classic “synchronize and suffer” and describes locks, concurrent collections, executors, atomic references etc. in more detail. This is what most of us typically deal with in our daily Java programming, and the book is a great introduction to those modern (if old, in a way) APIs.

That’s about one third of the book. The rest is devoted to much more interesting, intriguing and powerful tools: software transactional memory and actors.

Sometimes we have to deal with shared mutability, and very often we need to coordinate many threads accessing many variables. The classic synchronization tools don’t have proper support for it: Rolling back changes and preventing one thread from seeing uncommited changes of another is difficult, and most likely they lead to coarse-grained locks which basically lock everything while a thread is mutating something.

We know how relational databases deal with it with their ACID transactional model. Software transactional memory is just that but applied to memory, with proper atomicity, consistency and isolation of transactions. If one thread mutates a transactional reference in transaction, another will not see it until that transaction is committed. There is no need for any explicit locks as the libraries (like Akka or Clojure) monitor what variables you access and mutate in transaction and apply locking automatically. They even can rollback and retry the transaction for you.

Another approach is isolated mutability, a.k.a. actors, best demonstated on Akka. Each actor runs in a single thread and all it can do is receive or pass messages. This is probably closest to the original concept of object-oriented programming (recommended reading by Michael Feathers). You have isolated cells that pass messages to each other, and that’s it. When you have a task to execute, you spawn actors and dispatch it to them as immutable messages. When they’re done, they can call you back by passing another message (if the coordinator is also an actor), or if you’re not that pure you can wait for the result. Either way, eveything is neatly isolated in scope of a single thread.

Lengthy as this summary/review is, it really does not do justice to the book. The book itself is dense with valuable information and practical examples, which are as close to perfection as possible: There are a few recurring problems which are fairly simple and easy to grasp, solved over and over again with different techniques and different languages. There are many examples in Java, Scala, Groovy, Clojure and JRuby, dealing with libraries such as the core Java API, Clojure, Akka, GPars… In a few words, a ton of useful stuff.

Last but not the least, it’s excellently written. If anyone has seen Venkat in real life, this book is all like him – entertaining, but also thought-provoking, challenging and inspiring. It reads like a novel (if not better than some of them) and is very hard to put down until you’re done.

Highly recommended.

IDEs of the Future

Clojure, Life, Programming,

I suspect that by now everyone has seen Bret Victor’s “Inventing on Principle” talk. If you haven’t, here it is:

Bret Victor – Inventing on Principle from CUSEC on Vimeo.

I found it great and inspiring not only for the interactive features, but also for the moral parts. I’ve seen some previous inventions by Bret in the past around WorryDream.com. I wish I had been taught science like this. I try to teach like this, and this definitely is the way I am going to try to teach my children with.

Back to the point, though. Cool as they were, Bret’s interactive examples made me ask myself whether it could work with something more complicated (or less contrived?). I was not quite sure.

Today Prismatic served me Chris Ganger’s Light Table – similar concept, even inspired by Bret, applied to an IDE. Take a look at this video:

The video tells a lot, but if you prefer text you can read more at Chris’ blog.

Now, this is something that probably could work. Maybe not exactly like showed here, maybe not that minimalistic, but it has some ideas that I would really like to see in my IDE today.

  • Showing and searching documentation this way seems absolutely possible. Eclipse does not do it for me yet, though.
  • I would really love to see this kind of view where you focus on a single chunk of code (class, method or function) and can automatically see all the referenced bits (and only them) and navigate back and forth. I imagine a navigable graph that takes me between places, but it shows contents of more than one file at a time, and is not really based on files but “chunks” like functions or classes. Does not seem very far either, and could be as life-changing as multiple desktops or monitors (if not more).
  • Finally, the interactive debugging. Looks great and could work on functional language like Clojure and I can see how it would work there. It would be one hell of an effort to get it working for Java though, with all the encapsulation and baroque ceremony.

All in all, very inspiring ideas. I really hope my IDE does some of those things one day. Keep up the good work, guys!

Ring Handlers – Functional Decorator Pattern

Clojure, Programming,

During our last pairing session with Jacek Laskowski on Librarian there was a brief moment of confusion over Ring handlers. We struggled for a short while trying to figure out what order to put them in and what it really means to have code like:

(def app
  (-> routes
    ; sandbar
    (auth/with-security security-policy log-in)
	; compojure helper that includes a few Ring handlers
    site
	; sandbar again
    session/wrap-stateful-session))

It didn’t take us long to figure it out and the solution turns out to be a very elegant functional flavor of decorator.

It’s easy to dive too deep without proper understanding (and that’s what I admittedly did). Let’s start from the very beginning and see what these bits really mean. For starters, here’s a very basic app in plain Ring that simply returns the entire request:

(defn my-handler [request]
    {:body (str request)})

(def app my-handler)

When I hit http://localhost:3000/?my_param=54 in my browser, in return I get:

{:remote-addr "0:0:0:0:0:0:0:1", 
 :scheme :http, 
 :request-method :get, 
 :query-string "my_param=54", 
 :content-type nil, 
 :uri "/", 
 :server-name "localhost", 
 :headers {"cookie" "__utma=111872281.60059650.1328613066.1328726201.1328785442.5; __utmz=111872281.1328613066.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)", "connection" "keep-alive", "accept-encoding" "gzip, deflate", "accept-language" "pl,en-us;q=0.7,en;q=0.3", "accept" "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "user-agent" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0", "host" "localhost:3000"}, 
 :content-length nil, 
 :server-port 3000, 
 :character-encoding nil, 
 :body #<Input org.mortbay.jetty.HttpParser$Input@7c04703c>}

Note that my_param made it to :query-string, but obviously it’s quite inconvenient at this point and that’s not what we really want to deal with.

What is app at this point? No magic here, it’s just a very simple and boring function.

Let’s move on and add one of the seemingly magic Ring handlers – ring.middleware.params/wrap-params:

(def app
  (wrap-params my-handler))

This time for the same URL I get the same map, with a few new entries:

{:remote-addr "0:0:0:0:0:0:0:1", 
 ; Trimmed for brevity
 :query-params {"my_param" "54"}, 
 :form-params {}, 
 :query-string "my_param=54", 
 :params {"my_param" "54"}}

I can see that the wrapper added a few new entries: :query-params, :form-params and :params. Great, just like it was supposed to.

Now, what is app at this point? Just like before, it’s a regular function of request. So what does wrap-params really do? Let’s take a peek at (parts of) its source:

(defn wrap-params
  [handler & [opts]]
  (fn [request]
    (let [request  (if (:query-params request)
                     request
                     (assoc-query-params request))]
      (handler request))))

assoc-query-params is no magic, it simply parses query params and merges it with the request map.

Now let’s take a close look at the last line and back at wrap-params signature. Here’s what’s really going on:

  1. wrap-params takes handler (which is a function of request) as argument. In our case, it’s the trivial function that returns request in body.
  2. It then performs some work, in this case rebinding request to a map with a few more entries.
  3. Eventually it calls the handler that it got as parameter with the augmented request map.

In other words, wrap-params takes a handler function, and returns a function that performs some extra work and invokes the original handler.

Does it look familiar? Yup, it’s the old good decorator pattern. Do some work, then pass control on to the next handler (which can also be a decorator and delegate it further). In this case, though, it’s astonishingly simple (compared to what it takes in Java).

Now let’s say I want to chain one more handler that relies on the previous one. Let’s say I dislike strings and want to map params by Clojure keywords. There’s a handler for it: ring.middleware.keyword-params/wrap-keyword-params.

No need to think too long, let’s jump in and use it:

(def app 
  (wrap-keyword-params (wrap-params my-handler)))

… and I get:

{; Trimmed for brevity
 :params {"my_param" "54"}}

Whoops, that’s not what I expected. wrap-keyword-params was supposed to create a map with keys as keywords, not strings. Why didn’t it work?

Naive intuition tells me to treat wrappers as function calls. I wrap my-handler in wrap-params and pass the result of this invocation to wrap-keyword-params, right? Wrong!

Take a look at a sample wrapper above (wrap-params) and think what we were trying to do. What I really created here is a reversed chain like:

  1. Given a request, map its :params into keywords (wrap-keyword-params).
  2. Then pass control to the next function in chain, wrap-params. It parses query string and adds :params map to request.
  3. Then pass control to my-handler which prints the whole thing to browser

Nothing happens in the first step, because :params does exist at this point – it’s only created by wrap-params in the second step.

If we reverse it, it works like expected:

(def app 
  (wrap-params (wrap-keyword-params my-handler)))

{; Trimmed for brevity
 :params {:my_param "54"}}

To recap, a few things to remember from this lesson:

  • In functional programming, the decorator pattern is elegantly represented as a higher order function. I find it much easier to grasp than the OO flavor – in Java I would need an interface and 3 implementing classes for the job, greatly limiting (re)usability and readability.
  • In case of Ring wrappers, typically we have “before” decorators that perform some preparations before calling the “real” business function. Since they are higher order functions and not direct function calls, they are applied in reversed order. If one depends on the other, the dependent one needs to be on the “inside”.

Java Puzzlers: Number.MIN_VALUE

Programming,

If I asked you what the largest possible Integer in Java is, you would easily answer 2^31-1, or Integer.MAX_VALUE. Smallest possible Integer? Integer.MIN_VALUE, -2^31.

How about a Long? It’s also a Number, and each Number has MIN_VALUE and MAX_VALUE, and you would be right to refer to them.

Obvious and boring, right?

How about a Double then?

It’s also a Number, and so it has Double.MAX_VALUE, and yes that is the maximum. Minimum? Let me go already, obviously Double.MIN_VALUE!

Nope.

I only learned it today even though I’ve been programming Java for years. Double.MIN_VALUE is actually a positive number. Quick look at docs says it all: It is the smallest positive nonzero value of type double. If you’re really looking for the smallest value of Double, you need to refer to Double.MIN_NORMAL.

Yes, docs are very clear about it. Yes, it is a static constant, so LSP does not really apply (as if Java didn’t violate it all the time). Yet I still think it looks like a duck, it quacks like a duck, but needs batteries. It has different semantics for no good reason, does not follow the principle of least surprise and is plain unintuitive. I add it to my shelf of quirks along with half of Swing, everything around Dates and Calendars, equals and… what am I still doing here by the way?