Monthly Archives: March 2012

33rd Degree 2012

After great success of the first edition of 33rd Degree I did not really need anyone to convince me to participate this year. I can honestly say that I’ve been waiting for it for a year, and it definitely was worth it.

Day 1

The conference opened with three keynotes. The first speaker was Raffi Krikorian from Twitter who spent good deal of time explaining what Twitter really is: A gigantic event pump that consumes massive amounts of messages and broadcasts them as fast as it can. He also provided a very coarse-grained look over some pieces of their architecture, including heavy use of parallel scatter-gather algorithms even for fairly basic tasks like rendering a timeline. Another interesting bit was their own Finagle library for asynchronous networking with all the fancy stuff of failure detection and failover/retry, service discovery, load balancing etc. Worth noting is that Raffi did not say a bad word about Rails, but only explained why it did not scale to their level.

The second keynote was Ken Sipe talking about the complexity of complexity. I am not sure who was first, but I’m under the impression that this talk was in many ways similar to Rich Hickey’s Simple Made Easy, except for that it was a lot more fluffy. Rich had a very clear point to make and he did it perfectly, with witty and merciless criticism of some flavors of agile as a little cherry on top.

The third keynote was Venkat Subramaniam on Facts and Fallacies of Everyday Software Development. Addressed to pointy-haired bosses (including inside each one of us), it challenged some common opinions like trusting “proven technologies” (read: fossils or large vendors) and advocated polyglot programming (he can’t help but do it even when he talks about “pure Java”). Hardly anything new, but Venkat’s beloved style still made it an entertaining talk.

After those keynotes we got smaller presentations in parallel tracks with some hard decisions to make. I wish I went to Matthew McCullough’s git workshop. Instead I wasted some time looking at JFrog presentation by Frederic Simon (I expected some interesting stuff on continuous integration & delivery). Sadek Drobi’s talk on Play and non-blocking IO was quite OK, but I did not really buy the product. It may be because I’m not into Play, but I found the solution somewhat overengineered and much less appealing than Finagle.

The last technical talk of the day for me was Wojciech Seliga’s piece on tests around JIRA. Those guys are quality nuts, but I can honestly say I would feel really safe and comfortable in this environment. He explained all kinds of unit tests that they have: Regular “boring” unit tests, integration tests (including talking to other Atlassian products), functional tests and platform tests. Even though most of the functional tests run against an API (and only some on Selenium), testing the whole system takes many hours on a 70-worker CI cluster, and they still do it on each and every commit. That is mostly because of platform and performance tests, where they meticulously test against all kinds of platforms: combinations of OS, architecture, DB and whatnot.

Wojciech openly admitted that sometimes it takes them days to make a build green, but they are still very rigorous about it. He also shared some good tricks. Tests are very neatly organized in categories and you can drill down to see what exactly is failing and why. Failing tests go to quarantine in order to keep developers from assuming that there is a false negative that can be ignored while there can actually be other issues making the build fail. Once something goes into quarantine it’s either fixed or eradicated, there is zero tolerance for failure.

Day 2

I started the second day with Barry O’Reilly’s talk on agile & lean startups. I expected much fluff, but I actually learned a few interesting things. We all now the agile/lean cycle of (rougly speaking) iteratively inventing, creating, testing and evaluating, and we got to hear about it here as well. What was interesting and new to me though was how much you can do in the market research field. Two good examples include: Create very detailed portraits of your potential users, to the point where you can actually imagine this person as if you walked their shoes, feel their needs, think how they may want to use your product etc, eventually leading to detailed sketches and scenarios. Consider several distinct portraits and see what functionaliy and user experience emerges. The second idea that sounds crazy to my introvertic self is actually going to the street, cafes etc. and asking strangers about their needs and opinions. Reportedly they love it (“Building new Facebook? I wanna be a part of it!”).

Then I listened to Matthew McCullough’s presentation on economic games in software (and life, actually). Perfectly prepared, rich and fluent, easily one of the best speakers I saw in action (even when he’s coding, like last year on git and hadoop). Matthew touched a few topics: Money is not the only (and not the most important) part of the equation. It may be better to live in a nice place with your family and friends around you than to be paid $300/hour and work on the South Pole. It goes the other way round as well: If you hire people, make sure you motivate them in other ways – create a great environment, listen to their needs etc. Speaking of money, he also explained some tricks that are used on us every day – such as shaping restaurant menu or software license pallette so that they have some ridiculously overpriced things that no-one would buy only to make you buy other overpriced (but cheaper) products.

Enough soft talks, next one was Venkat Subramaniam talking about concurrency, reportedly in pure Java. Started off with a seemingly trivial synchronization issue that is very hard to do right in Java and easily generates a ton of boilerplate. Then he went on to resolve it in a few lines of Clojure (surprise, surprise), explained the concept of STM and went back to Java, resolving the original problem using this little concurrency library (yes, Clojure in Java). Very witty and entertaining, and even though I know some Clojure I wasn’t aware how easy it is to get it to work for you in Java. It really can be just an accessible concurrency library!

Next on my schedule was Sławek Sobótka’s talk on basic software engineering patterns. I enjoy most of the works that he publishes on his blog and the live presentations. This was no exception, even if most of the stuff wasn’t new to me. In short, it was a gentle introduction to some ideas behind DDD and modeling. From basic building blocks (value objects, entities, aggregates) up to operations (services) and customization (policies), all neatly organized in clear layers and levels. Professional, clear, fluent and to the point.

Then I saw Ken Sipe’s presentation on web security, which was basically a run through OWASP Top Ten threats. Unlike the keynote this one was little fluff much stuff and taught me something about common security issues. I knew most of the threats, but never really went through the OWASP list myself. Very good and thorough presentation, complemented by personal stories and interesting digressions.

Venkat Subramaniam showing Scala in a terribly overcrowded room was the last thing on my agenda for the day. I’ve never done anything in this language and I found this presentation a very gentle and interesting introduction. Looks like something between powerful but chaotic Groovy (that feels more like a scripting language to me) and old and baroque Java. I could actually give it a try, though I am yet to think of an applicable project.

Day 3

I kicked off the last day with Jacek Laskowski’s talk on Clojure. It clearly was not possible to show all the interesting corners of the language, but I think he did a good job at explaining the very basics as well as some concurrency concepts. It could have included more details or examples on why we would want to use those concurrency constructs, but still it managed to spark some interest (and not scare people off), and that alone is a big win.

Then I listened to Bartosz Majsak talking about Arquillian – a JBoss library for running higher level tests (demonstrated on Selenium-driven acceptance tests) using mock classes in a managed container. Quite interesting, I may give it a try when I get to play with JEE webapps.

The last strictly technical talk of the day was Nate Schutta on HTML5. Nate asked the audience which parts of the stack they are interested in, and after the introduction went over what we were most interested in: Canvas, client-side data storage and worker threads. Very good, fluent talk, explaining the foundations and history behind HTML5 and complemented by practical examples.

Finally we got three keynotes. The first one was Nate Schutta on code craft. Just like you may expect from keynote, it was a mile wide and an inch deep talk about code and work quality. Keep it simple, keep it short, write tests, don’t reinvent the wheel, get and respect feedback… you know it. Professional and interesting, but immediately forgettable. Things to take away: PMD, Clover – run them regularly, radiate information and encourage others to participate in improvement. Culture kills.

Second keynote was Jurgen Appelo telling us how to change the world. It was very well prepared, starting with a witty and masochistic self-introduction, but then it continued to explain a 4-tier model of influencing people:

  • Work iteratively and reflect on your progress, for example using the classic Plan-Do-Check-Act approach.
  • Understand and drive individuals. Make them aware to the idea or product. Make them desire it by making it challenging, interesting or in other ways appealing. Finally pass on knowledge and abilities.
  • Affect social interactions and culture. Understand that each successful adoption has several stages: Innovators and early adopters who are personally interested in getting your product or idea (crazy people who are ready to drive around the city looking for the last iPad). The majority or skeptics whose needs are largely different (more interested in the ability to watch a movie on the train than in having the latest and coolest hardware). Finally the laggards who can undo your efforts if you stop early.
  • Shape the environment to reinforce all of the above: Radiate information, create a strong cultural identity, use incentives, infrastructure and institutions that encourage or guard the right direction.

There was much more to it than this shallow summary, and there is a reason why it fits in a 19-page booklet (downloadable).

The last keynote was Uncle Bob Martin demanding professionalism. Let me sum it up. Started off with a weird off-topic discussion on why people see more colors than dogs and les than bees. Then came a good part: Creating software resembles engineering, but not the way we popularly believe. “Regular” engineers and architects create very detailed documents and are relatively cheap compared to the manufacturing and execution. It’s often said that programmers are on the “manufacturing” stage, while actually everything they do is on engineering. They actually produce very detailed documents (source code), and that’s a huge effort compared to execution (compilation and running). So the cost model is inverted.

Then Uncle Bob slowly went to the point. Our computers are many orders of magnitude more powerful than those of the distant past, yet we use very similar constructs. Execution on modern hardware is practically instant and free. Uncle Bob asked where it got us, and someone replied “Angry Birds”. To this point I very much agree: It’s extremely sad how we waste all the power on bloated frameworks or worthless entertainment. But the point that Uncle Bob tried to make was quite different: Since hardware and execution is so cheap, we need to keep the model inverted by making execution as fast as possible, and that is professionalism. Write tests, don’t test through GUI, avoid database.

Let me repeat: Hardware is blazingly fast, so our primary concern should be to keep the build fast so that most of the time is spent on coding, not compiling deploying. Am I the only one who finds it self-conflicting? The issue is not that we create useless junk. It’s not that we miss business requirements. It’s not that we suck at estimation. It’s to keep the cost model inverted. Actually, when Uncle Bob encouraged the audience to ask questions, someone asked: “How to estimate better?” At this point he laughed and left the stage.

One thing worth taking away: Good architecture is what makes decisions about infrastructure deferrable to the last stages of the project. Keep expanding the core and mocking if necessary, and think of database or frameworks last.

Final Word

All in all, I would say this edition was as good as the last one if not better. I paid more attention to technical talks and some of them paid off. One thing that was definitely worse: It was terribly overcrowdeed! It was next to impossible to walk the halls or even the large exhibition area, not to mention lunch or bathrooms. I don’t think making it larger in the same venue was a good idea.

“Enterprise Integration Patterns”

Today messaging is a common tool, but for some reason it also does not seem to be getting much attention. It seems that blogs and schools are busy with OO (how to do it (badly) & let me show you how to it Right), languages, libraries, and other such low-level stuff. On the other hand, messaging has a very strong position on the market and potential employers are more likely to ask you about it than talk about all the 50 libraries in your CV.

Before plunging headfirst into coding I was looking for some solid semi-theoretical introduction that would explain what exactly messaging is, what problems it tries to solve and what exactly you can do with it. I found Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregory Hohpe and Bobby Woolf and decided to give it a try.

The book starts with a gentle introduction to enterprise integration and its challenges and compares four solutions: shared database, file exchange, RPC and messaging. It explains why messaging is sometimes superior to the alternatives (but is fair about the cost).

The next part presents a high-level overview of the messaging concepts: channels, messages, pipes & filters, routers, translators and endpoints. It also shows a non-trivial (but very clear and interesting) example application which demonstrates where each of the pieces fits.

Finally, it goes through each of the concepts in detail, presenting a handful of patterns for each. This third part appears to be a very rich and complete catalog of common patterns, ranging from basic channel design (point-to-point vs. publish-subscribe, datatype channels, guaranteed delivery etc.), through message construction (command vs. document vs. event, request-reply etc.), routing (splitting and aggregation, content-based routing, dynamic routing etc.), and so on.

Each pattern is presented in a similar way. Authors sketch a problem, discuss possible solutions with their pros and cons, and finally present the pattern as the proposed common way to address it. After the discussion many patterns also have code examples in JMS, MSMQ in .net or TIBCO. Between the chapters there also are a few coding “interludes” with a larger application using several patterns together.

I loved this book. It was well thought over. It’s clear from the first to the last page that the authors knew what they want to convey and masterfully designed a book that probably reads very well on all levels, from absolute beginner to someone who has already worked with messaging. It’s not intimidating to beginners, nor is it a shallow entry-level piece that will make everyone else yawn.

It’s not only a “dry” catalog of patterns. Everything is provided in broader context and it really shows how those pieces work together. It’s a very comprehensive introduction to messaging that you can actually use to learn the subject (nothing like trying to learn OO from reading the Gang of Four). On the other hand, the catalog of patterns is very rich and I am quite sure it satisfies most or even all of the needs of real-world applications.

I haven’t read too many books on messaging, but this one seems so good that if you are even slightly interested in the topic, it is a must read.