Tag Archives: Books

“Growing Object-Oriented Software, Guided by Tests” (Book Review)

Posted on by
Reply

“Growing Object-Oriented Software, Guided by Tests” by Steve Freeman and Nat Pryce has been on my to-read list ever since I saw Steve at 33rd Degree 2011. Even though I did not really like the presentations, somehow I became intrigued enough.

The first two parts of the book explain what object-oriented programming and test-driven-development are. It explains the “tell, don’t ask”, encapsulation and information hiding, some pieces of design and architecture (ports and adapters). It contains a short explanation of what TDD is (the basic 3-phase cycle known to everyone) and extends it beyond unit tests, up to blackbox integration tests.

The third part is a long case study – writing a real, nontrivial application with Swing GUI, XMPP and interesting domain. The study is pretty long, and even though I’m not a fan of them this one is just perfect. It’s very easy to follow, taking small well-explained steps. It’s a rich and very practical example of TDD walking hands in hands with elegant object-oriented design.

The fourth part is a long and interesting catalog of “test smells” – shows the links between test complexity and readability and quality of covered production code; tells how to write elegant, useful tests and assertions; gives advice on how to write tests that won’t be too constraining in future; and more. The last part explores some practical sides of testing in multithreaded environment and around persistence.

I loved the book! It’s an awesome piece of work that covers so many topics in such a clear, deep and interesting way that I would call it a must read for everyone. It’s a great book on testing, but not just that. It shows where testing fits in the software development process and explains the very important link to object-oriented design. It’s full of small, hidden gems (not only about software testing itself) that I will remember for a long time. At the same time it’s also very practical, pragmatic and concise.

I think it has much to offer to people of any skill level (perhaps except for novice). Even though I have been doing OOP and TDD for years, I really enjoyed it and I believe I learned much new stuff.

If you haven’t done so yet, do yourself a favor and go read it now!

“ClojureScript Up and Running” (Book Review)

Posted on by
2

I’ve recently finished the “ClojureScript Up and Running” book by Stuart Sierra and Luke VanderHart. Here’s a quick review of it.

It opens with a quick introduction which attempts to present ClojureScript as the alternative to (or a matured “version” of) JavaScript. Then it immediately dives into gory technical details:

  • It shows how to set up a Leiningen project with lein-cljsbuild and explains the compilation process in detail (with all the possible parameters and modes).
  • It explains the development process – getting ClojureScript, working with browser REPL, testing, packaging for use with CLJS and plain JS applications.
  • It gives a basic introduction to the language, which really is a head-first guide to Clojure (though I suspect it’s far from enough for people who don’t know Clojure, and for those who do know it it’s no use).
  • It explains integration with JavaScript, but mostly on the level of Google Closure – exporting functions and namespaces for the world outside, or using external libraries with Closure advanced compilation.
  • It explains integration with Clojure, especially using EDN as an alternative to JSON.
  • It also introduces a few CLJS libraries (little code, just an idea of what a library does). Among others it includes C2, jayq, enfocus, core.logic, domina. Google Closure is on the list as well.

    •  

    There are many things I missed, though.

    First and foremost, it does not do enough to explain why we need ClojureScript. It does not really try to convince anyone, it’s pretty much a very technical “up and running” guide.

    It does not tell you how to write ClojureScript code. The explanation of setup, compilation and integration is great, but that’s all there is. Actually, there’s little about ClojureScript itself in the book. It’s like (unconvincingly) telling you that Java is the more robust and productive alternative to C++, showing how to use javac and Eclipse, “Hello World”, and a brief indication that there’s also JEE and Swing and this and that. I wish it had some case studies, chunks of actual ClojureScript code, some discussion of architecture, patterns, sample applications…

    One more thing I did not like is that it apparently ignores the JavaScript world (except for Google Closure). There are many rich and mature JavaScript libraries out there that solve many problems, and this book has not a word on the common ways to integrate them. I spent a good while experimenting with CLJS with Backbone, Knockout, Angular and jQuery, and it’s quite a difficult, frustrating task. I know I can write a CLJS library to do the same things that Knockout does, but I would prefer to learn how to integrate the existing library with my CLJS application to solve a real problem, or be introduced to a pure CLJS alternative that someone has already created.

    Perhaps it reflects the current state of ClojureScript – not yet very mature or stable, without mature and established libraries, patterns.

    All in all, I have mixed feelings. The book is very dense and concrete, and delivers much content on so few pages. It’s true to its title – gets you up and running, and does it very well. But then immediately leaves you alone in the woods. We are yet to see “The ClojureScript Book”.

“Release It!”

Posted on by
Reply

A while ago I wrote a post on Learning to Fail inspired largely by Michael T. Nygard’s book titled “Release It”. Now it’s time to review the book itself.

As the sub-title says, the book is all about designing and deploying production-ready software. It opens with a great introduction on why it really matters: Because software often is critical to business. Because its reliability and performance is really our job and matter of professionalism. Finally (if that’s not enough), because its behavior in production will have huge impact on our quality of life as well – matter of choosing between panic attacks and phone ringing at 4 AM, or software Just Working by itself, letting you enjoy healthy life and doing more fun stuff at work. That’s the center of mass here, by the way: More on development and operations, less on management and business.

The book is divided into four main areas. Each starts with a bit of theoretical introduction and/or an anecdote, followed by discussion of concrete phenomena, problems and solutions. Even though it might appear as a collection of patterns and antipatterns, it’s much more than that. Patterns and antipatterns are just a form, but it’s really about setting the focus for a few pages and naming the problem. Anyway, the “pattern” and “antipattern” concept is gone by the middle of the book.

The first part talks about stability, and how it’s impacted by error propagation, lack of timeouts, all kinds of poor error handling, weaker links etc. Then it shows solutions: How to stop errors from propagating. How to be paranoid, expect failure in each integration point (with 3rd party and not), and deal with them. How to fail fast. And so on.

The second part talks about capacity: Dealing with load, understanding constraints and making predictions. Impact from seasonal phenomena or ad campains. Strange and not obvious usage patterns – hitting “refresh” button, web scrapers etc. Finally, dealing with those issues with proper use of caching, pooling, precomputing content and tuning garbage collection.

The third part is a bag with all kinds of design issues: networking, security, availability (understanding and defining requirements, followed by load balancing and clustering), testing and administration.

The last part is all about operations: logging, monitoring, transparency, releasing, that kind of stuff. How to organize it so that routing maintenance will be less pain, monitor will let us detect issues early, and finally after or during an issue we will have enough information to diagnose it.

Some problems are discussed from bird’s eye view. Most problems are more down-to-earth, providing detailed discussion of an issue with a sketch for solution with its weak and strong points. Finally, when applicable, author rolls up his sleeves and is ready to talk about concrete code, SQL, heapdumps, scripting etc.

The book is actually full of real war stories, anecdotes, code samples, tool descriptions, case studies, and all kinds of concrete content. There are a few larger stories that go on like this: On this project the team did this, this and that in order to migate such and such risks. When marketing sent an advert, or when the system was launched, or during routine maintenance, this and this broke and started causing problems. We did heapdumps, monitored traffic and contents, read or decompiled the code etc. and discovered problems there and there. Finally, we solved them with this and that. And here comes the detailed list of trouble spots and ways to mitigate them. It’s really a complete view – from business perspective and needs, down to nitpicking about particular piece of code or discussing popular tools.

Apart from being a great collection of real problems and tricks, there is one longer lasting, recurring aspect that may be the most valuable lesson here. Michael T. Nygard regularly shows (and makes you feel it deep in your guts, especially if you did some maintenance in production) that you really should be expecting failure everywhere and every time. You should try and predict, and mitigate, as many issues as possible, as early as possible. You should be paranoid. More than that, embrace the fact that you will fail to predict everything, and design so that even random unpredictable failures won’t take you down and may be easier to solve.

All the time it’s very concrete and complete. It also feels very professional, genuine and even inspiring.

Highly recommended.

Learning to Fail

Posted on by
Reply

Back at university, when I dealt with much low-level problem solving and very basic libararies and constructs, I learned to pay attention to what can possibly go wrong. A lot. Implementing reliable, hang-proof communication over plain sockets? I remember it today, a trivial loop of “core logic” and a ton of guards around it.

Now I suspect I am not the only person who got used to all the convenient higher-level abstractions so much that he began to forget this approach. Thing is, real software is a little bit more complex, and the fact that our libraries kind of deal with most low-level problems for us doesn’t mean there are no reasons to fail.

Software

As I’m reading “Release It!” by Michael T. Nygard, I keep nodding in agreement: Been there, done this, suffered that. I’ve just started, but it’s already shown quite a few interesting examples of failure and error handling.

Michael describes a spectacular outage of an airline software. Its experienced designers expected many kinds of failures and avoided many obvious issues. There was a nice layered architecture, with proper redundancy on every level from clients and terminals, through servers, through database. All was well, yet on a routine maintenance in database the entire system just hung. It did not kill anyone, but delayed flights and serious financial losses have an impact too.

The root cause turned out to be one swallowed exception on servers talking to the database, thrown by JDBC driver when the virtual IP of the database server was remapped. If you don’t have proper handling for such situations, one such leakage can lock the entire server as all of its threads wait for the connection or for each other. Since there were no proper timeouts anywhere in the server or above, eventually everything hung.

Now it’s easy to say: It’s obvious, thou shalt not swallow exceptions, you moron, and walk on. Or is it?

The thing is, an unexpected or improperly handled error can always happen. In hardware. Or a third party component. Or core library of your programming language. Or even you or your colleague can screw up and fail to predict something. It. Just. Happens.

Real Life

Let’s take a look at two examples from real life.

Everyone gets in the car thinking: I’m an awesome driver, accidents happen but not to me. Yet somehow we are grateful for having airbags, carefully designed crumple zones, and all kinds of automatic systems that prevent or mitigate effects of accidents.

If you were offered two cars at the same cost, which would you choose? One is in pimp-my-ride style with extremely comfortable seats, sat TV, bright pink wheels and whatever unessential features. But it breaks down every so often based on its mood or the moon cycle, and would certainly kill you if you hit a hedgehog. The other is just comfortable enough, completely boring, no cool features to show off at all. But it will serve you 500,000 kilometers without a single breakdown and save your life when you hit a tree. Obvious, right?

Another example. My brother-in-law happens to be a construction manager at a pretty big power plant. He recently took me on a trip and explained some basics on how it works, and one thing really struck me.

The power station consists of a dozen separate generation units and is designed to survive all kinds of failures. I was impressed, and still am, that in power plant business it’s normal to say stuff like: If this block goes dark, this and this happens, that one takes over, whatever. No big deal. Let’s put it in a perspective. A damn complicated piece of engineering that can detect any potentially dangerous conditions, alarm, shut down and fail over just like that. From small and trivial things like variations in pressure or temperature, through conditions that could blow the whole thing up. And it is so reliable that when people talk about such conditions, rare and severe as they are, they say it in the same tone as “in case of rain the picnic will be held at Ms. Johnson’s”.

Software Again

In his “After the Disaster” post, Uncle Bob asked: “How many times per day do you put your life in the hands of an ‘if’ statement written by some twenty-two year old at three in the morning, while strung out on vodka and redbull?”

I wish it was a rhetorical question.

We are pressed hard to focus on adding shiny new features, as fast as possible. That’s what makes our bosses and their bosses shine and what brings money to the table. But not only them, even we (the developers) naturally take most pride in all those features and find them the most exciting part of our work.

Remember that we’re here to serve. While pumping out features is fun, remember that those people simply rely on you. Even if you don’t directly cause death or injury, your outages can still affects lives. Think more like a car or power station designer, your position is really closer to theirs than to a lone hippie who’s building a little wobbly shack for himself.

When an outage happens and also causes financial loss, you will be to blame. If that reasoning does not work, do it for yourself – pay attention now to avoid pain in future, be it regular panic calls at 3 AM or your boss yelling at you.

More Stuff

Michael T. Nygard ends that airline example with a very valuable advice. Obvious as it may seem, it feels different if you realize it and engrave it deep in your mind. Expect failure everywhere, and plan for it. Even if your tools handle some failures, they can’t do everything for you. Even if you have at least two of each thing (no single point of failure), you can still suffer from bad design. Be paranoid. Place crumple zones on every integration point with other systems, and even different components of your system, in order to prevent cracks from propagating. Optimistic monoliths fail hard.

Want something more concrete? Go read “Release It!”, it’s full of great and concrete examples. There’s a reason why it fits in a book and not in a blog post.

“Programming Concurrency on the JVM”

Posted on by
Reply

A few years ago when I took concurrency classes pretty much everything I was told was that in java synchronized is key. That’s the way to go, whenever you have multithreading you have to do it this way, period. I also spent quite a while solving many classic and less classic concurrency problems using only this construct, reimplementing more fancy locks using only this construct, preventing deadlocks, starvation and everything.

Later in my career I learned that is not the only way to go, or at least there are those fancy java.util.concurrent classes that take care of some stuff for you. That was nice, but apparently I never took enough time to actually stop and think how those things work, what they solve and why.

The light dawned when I started reading Programming Concurrency on the JVM: Mastering Synchronization, STM, and Actors by Venkat Subramaniam.

The book starts with a brief introduction on why concurrency is important today with its powers and perils. It quickly moves on to a few examples of different problems: IO-intensive task like calculating size of a large directory, and computationally intensive task of calculating prime numbers. Once the ground is set, it introduces three approaches to concurrent programming.

The first way to do it is what I summed up in the first paragraph, and what Venkat calls the “synchronize and suffer” model. Been there, done that, we know how bad it can get. This approach is called shared mutability, where different threads mutate shared state concurrently. It may be tamed (and a few ways to do it are shown in the book), but is a lot harder than it seems.

Another approach is isolated mutability, where each mutable part of state is only accessed by one thread. Usually this is the actor based concurrency model. The third way is pure immutability where there simply is no mutable state. That is typical for functional programming.

In the following chapters the book explores each of those areas in depth. It briefly explains the Java memory model nad shows what options for dealing with shared mutability and coordinating threads exist in core Java. It clearly states why the features from Java 5 are superior to the classic “synchronize and suffer” and describes locks, concurrent collections, executors, atomic references etc. in more detail. This is what most of us typically deal with in our daily Java programming, and the book is a great introduction to those modern (if old, in a way) APIs.

That’s about one third of the book. The rest is devoted to much more interesting, intriguing and powerful tools: software transactional memory and actors.

Sometimes we have to deal with shared mutability, and very often we need to coordinate many threads accessing many variables. The classic synchronization tools don’t have proper support for it: Rolling back changes and preventing one thread from seeing uncommited changes of another is difficult, and most likely they lead to coarse-grained locks which basically lock everything while a thread is mutating something.

We know how relational databases deal with it with their ACID transactional model. Software transactional memory is just that but applied to memory, with proper atomicity, consistency and isolation of transactions. If one thread mutates a transactional reference in transaction, another will not see it until that transaction is committed. There is no need for any explicit locks as the libraries (like Akka or Clojure) monitor what variables you access and mutate in transaction and apply locking automatically. They even can rollback and retry the transaction for you.

Another approach is isolated mutability, a.k.a. actors, best demonstated on Akka. Each actor runs in a single thread and all it can do is receive or pass messages. This is probably closest to the original concept of object-oriented programming (recommended reading by Michael Feathers). You have isolated cells that pass messages to each other, and that’s it. When you have a task to execute, you spawn actors and dispatch it to them as immutable messages. When they’re done, they can call you back by passing another message (if the coordinator is also an actor), or if you’re not that pure you can wait for the result. Either way, eveything is neatly isolated in scope of a single thread.

Lengthy as this summary/review is, it really does not do justice to the book. The book itself is dense with valuable information and practical examples, which are as close to perfection as possible: There are a few recurring problems which are fairly simple and easy to grasp, solved over and over again with different techniques and different languages. There are many examples in Java, Scala, Groovy, Clojure and JRuby, dealing with libraries such as the core Java API, Clojure, Akka, GPars… In a few words, a ton of useful stuff.

Last but not the least, it’s excellently written. If anyone has seen Venkat in real life, this book is all like him – entertaining, but also thought-provoking, challenging and inspiring. It reads like a novel (if not better than some of them) and is very hard to put down until you’re done.

Highly recommended.

“Enterprise Integration Patterns”

Posted on by
Reply

Today messaging is a common tool, but for some reason it also does not seem to be getting much attention. It seems that blogs and schools are busy with OO (how to do it (badly) & let me show you how to it Right), languages, libraries, and other such low-level stuff. On the other hand, messaging has a very strong position on the market and potential employers are more likely to ask you about it than talk about all the 50 libraries in your CV.

Before plunging headfirst into coding I was looking for some solid semi-theoretical introduction that would explain what exactly messaging is, what problems it tries to solve and what exactly you can do with it. I found Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregory Hohpe and Bobby Woolf and decided to give it a try.

The book starts with a gentle introduction to enterprise integration and its challenges and compares four solutions: shared database, file exchange, RPC and messaging. It explains why messaging is sometimes superior to the alternatives (but is fair about the cost).

The next part presents a high-level overview of the messaging concepts: channels, messages, pipes & filters, routers, translators and endpoints. It also shows a non-trivial (but very clear and interesting) example application which demonstrates where each of the pieces fits.

Finally, it goes through each of the concepts in detail, presenting a handful of patterns for each. This third part appears to be a very rich and complete catalog of common patterns, ranging from basic channel design (point-to-point vs. publish-subscribe, datatype channels, guaranteed delivery etc.), through message construction (command vs. document vs. event, request-reply etc.), routing (splitting and aggregation, content-based routing, dynamic routing etc.), and so on.

Each pattern is presented in a similar way. Authors sketch a problem, discuss possible solutions with their pros and cons, and finally present the pattern as the proposed common way to address it. After the discussion many patterns also have code examples in JMS, MSMQ in .net or TIBCO. Between the chapters there also are a few coding “interludes” with a larger application using several patterns together.

I loved this book. It was well thought over. It’s clear from the first to the last page that the authors knew what they want to convey and masterfully designed a book that probably reads very well on all levels, from absolute beginner to someone who has already worked with messaging. It’s not intimidating to beginners, nor is it a shallow entry-level piece that will make everyone else yawn.

It’s not only a “dry” catalog of patterns. Everything is provided in broader context and it really shows how those pieces work together. It’s a very comprehensive introduction to messaging that you can actually use to learn the subject (nothing like trying to learn OO from reading the Gang of Four). On the other hand, the catalog of patterns is very rich and I am quite sure it satisfies most or even all of the needs of real-world applications.

I haven’t read too many books on messaging, but this one seems so good that if you are even slightly interested in the topic, it is a must read.

Apprenticeship Patterns

Posted on by
Reply

Apprenticeship Patterns. Guidance for the Aspiring Software Craftsman by Dave Hoover and Adewale Oshineye has been lying on my bookshelf for quite a while. I expected a largely repetitive, buzz-driven soft read. It turned out to be quite a surprise.

Yes, Apprenticeship Patterns is a book about software craftsmanship. However, it’s very far from what wannabe-craftsmen say: “We’re all artists, why don’t people admire us like we deserve it.” Actually, it’s the exact opposite.

Learning is a very long road. Accept your ignorance as you begin, seek sources to learn from, kindred spirits or mentors. Get your hands dirty and learn by practice. Work with people, code breakable toys, read constantly, and practice, practice, practice. It’s only your responsibility to learn and improve your skills, diversify, and deepen your knowledge. Don’t repeat the buzz, but get your hands dirty and get to work. Finally, share what you learned with those behind you on the path and create communities where people can motivate each other and learn together. And don’t lose your motivation and goals along the road.

Now, the previous paragraph does look pretty fluffy. That’s just a birds-eye view, and those tend out to lack detail. The book itself is very down-to-earth, “dirty” and concrete. It’s an inspiring collection of thoughts, ideas and tricks on self-improvement.

I’ve seen quite a few talks and read a few articles on craftsmanship, and none of them was anywhere near as concrete and complete as this book. It doesn’t only apply to software. In fact, I believe it largely applies to any other area that involves learning in our life.

Highly recommended.

The Joy of Clojure: Learning “The Clojure Way”

Posted on by
Reply

The Joy of Clojure (cover)

Very often the journey with Clojure starts like this. You notice its growing popularity or want to attach a new tool to your belt. So you go through some quick introduction, learn the basics, get a few programs running… And eventually ask yourself: Fine, but what’s next? What’s the difference? How does one really work with Clojure?

That’s where The Joy of Clojure by Michael Fogus and Chris Houser enters the play. The goal of this book is to provide answers to these fundamental questions and teach you “The Clojure Way”.

It starts with an explanation of what Clojure is and the problems that it solves. Then it provides an overview of the language: data types, functions, collections, destructuring, composite data types and lazy evaluation. Finally it explores more advanced concepts: functional programming, metaprogramming, performance, mutation, concurrency and parallel programming.

However, by no means is it an average reference / language overview book. What it really does is explain the philosophy of LISP, functional programming and last but not least Clojure, and only then discuss all the technical stuff.

Just because the primary focus is not syntax and low-level technical details does not mean they are not explained in depth. Nothing is missing: syntax, performance, intricate details of data types and other constructs – they’re all here.

That’s where the book really shines. It’s easy to write a manual that describes a technology, but fails to describe its purpose. “The Joy of Clojure” managed to explain what Clojure really is, with all its fundamentals and idioms.

The major downside of the book is that some of the advanced low level technical / syntactical aspects are explained in a rather steep, if not discouraging way. Take macros for example. The concept is clear, but the examples could be less complicated and it falls short of explaining the basic syntax.

If you’re looking for an easy basic introduction to Clojure, or a practical guidebook on libraries, developing web apps etc. then probably this is not the best choice. But if you are up for a challenge or know some Clojure and are asking yourself the fundamental questions from the first paragraph of this review, you are going to love it. Sometimes it is steep and demanding, it may require you to reread some of the chapters later, but it definitely is worth it.

Note: This review is based on an early access edition provided by Manning Publications Co. The estimated publishing date is December 2010, but you can get a pre-release PDF already from Manning Publications.