Category Archives: Science

Strange Loop 2013

Clojure, Programming, Science

This year I had the pleasure to be at the Strange Loop conference in Saint Louis. Here’s a brief recap of what was happening there.

Preconf: Emerging Languages Camp

On the first day I sat in about a dozen presentations related to “emerging languages”. The good ones:

  • Daniel Gregoire – Gershwin: stack-based, concatenative Clojure. There were a few presentations of stack-based languages here, but in my opinion this one was the best. It’s been the first time I actually heard about such languages. Not very practical, but quite an interesting excercise none the less.
  • Bento Miso – Daimio: a language for sharing. An attempt at creating a language for creating web plugins, with sandboxing, clear and isolated execution spaces, interactions defined in in/out ports… A bit too hard to follow though..
  • Andreas Rumpf – Nimrod – very good presentation of quite interesting and well thought out language intended mainly for game development. Reportedly fast, very expressive, decent compiler with macros and some serious compile time optimizations… Worth a look.
  • Walter Wilson – Axiomatic Languages. Interesting, very theoretical approach to programming. Define programs in terms of axioms and logical reasoning. Want to sort a list? No problem, just define what it means for elements to be ordered and what a permutation is, then say that the desired output is a permutation that is ordered. Rather academic.
  • Tracy Harms – J. The joke says it’s been an emerging language for 23 years. I’ve never been able to get it (not that I tried much), and this presentation was a very good introduction. It included and decently explained a lot of the syntax as well as the underlying flow and concepts. Quite unusual and interesting.
  • Bodil Stokke – BODOL. This gem was one of the wildest things that I’ve seen not only on this conference, but also in my professional life. Ponies, pattern matching, logic programming, lambdas, monads, types, parsing, programming language theory, live coding… All running from Emacs.

I also attended two unsessions.

  • One was a live recording session of the Think Distributed podcast. 4 smart folks took questions from audience and shared their thoughts on testing, programming, handling failures etc. in distributed environment. Worthwhile.
  • The other un-session was largely based on the she++ documentary on bringing women to programming (and technology in general). Important problem that reappeared a few more times at the conference. It’s worth mentioning, though, that Strange Loop had an unusually high number of female speakers. They might have been preferred in some ways, but nonetheless delivered many awesome talks and the gender did not really matter.

Day One

  • Jenny Finkel – Machine Learning for Relevance and Serendipity. In this opening keynote we got to learn some of the challenges behind Prismatic. (If you haven’t seen Prismatic yet, do it now. It’s awesome.). Very good insights on the discovery problem, machine learning, collective intelligence, user behavior, statistics…
  • Jason Gilman – Visualization Driven Development. One more stab at the problem of using visual aids in programming. This time more like: Tired of waiting for the tools? Roll your own, very custom for the code you’re currently working with.
  • Philipp Haller – Scala Async. I don’t really do Scala, but I liked this presentation a lot. Solid, elegant presentation of upcoming features of Scala for async programming (inspired by F# and C#). Lots of good code, easy to follow examples and great explanation of the matter.
  • Craig Andera – Real-World Datomic. A little bit of a case study, much insight into design and usage of the database. We got to learn about designing and representing the domain model, all the index types and their use cases and some other internals of Datomic. Great talk.
  • Rich Hickey – Clojure core.async. Very, very good presentation not only on core.async (which I’ve never actually used before), but also on asynchronicity in general, CSP, performance, code expressiveness and cohesion… Not so much syntax and case studies, more of a solid explanation of reasons and usage of the library. Plus Rich is as brilliant a speaker as he is a creator.
  • Craig Muth – Xiki: GUI and Text Interfaces Are Converging. Demonstration of a nice… engine (?) for doing all kinds of things from plain text files or just a text editor: GUI prototyping, using git, databases, rails, file system navigation. Very impressive demos, but is it practical? Seems like it may need a good deal of Emacs-fu or a few dozen keyboard shortcuts to do effectively.
  • Joseph Wilk – Creative Machines. The presenter shared many different attempts to get the computer to create music (as well as some examples of work done by others on this topic). All this trying to answer whether machines can be creative, what it means to be creative, how does human creativity work and whether it can be reproduced by a machine. Passionate, well-delivered and thought-provoking, even though I do not fully agree with the conclusions.
  • Jen Myers – Making Software Development Make Sense to Everyone. Very passionate keynote on encouraging women to technology. Talking about iconic heroines of the past (Sally Ride, Nichelle Nichols), as well as something closer to our field and time. Jen presented the “little”, things that she is directly involved with at the moment (Dev Bootcamp, Girl Develop It), as well as shared her vision of a future where hopefully it no longer will be necessary.

At night there was a wild party in the City Museum. Warehouse turned to museum with lots and lots of industrial or fossilized attractions: High and twisted slides, caves, hanging bus, ball pit, crazy places to crawl and walk and lie and sit everywhere (ceilings, floors, going in all directions). Everything adult sized, and boy were those geeks happy as kids to do all this stuff. Last but not least, it was accompanied by very appropriate cave/jazz/dubstep band Moon Hooch. Words can’t really describe it, you had to experience it.

Day Two

  • Martin Odersky – The Trouble With Types. The first half of this keynote was a dense study of type systems in various languages, including the tradeoffs of dynamic vs. static and strong vs. weak. Martin did a great job explaining why Scala is looking the way it is, and why he thinks a strong, static type system is the way to go. The second half delved deep into Scala and to me was too abstract or needed too deep knowledge about Scala. Still, Odersky is a great speaker and thinker. Bonus points for the joke that Rich Hickey could take those slides and give the same talk and treating us to Bach.
  • Avi Bryant – Add all the things: Abstract algebra meets analytics. Very good presentation with very practical examples of how thinking in basic mathematical terms can make software development easier. It showed how you can apply the group theory to data aggregation and storage. It also had a few interesting tricks generally related to the area of data collection and analytics. Great presenter.
  • Crista Lopes – Exercises in Style. Take one problem (word count in text file) and solve it in one language in several different styles, every time deliberately taking some very specific constraints. Spaghetti script, procedural, functional, continuation passing style… Great exercises, worth a look (much bigger book coming out shortly). Unfortunately, was too slow in the beginning and had to cut a few examples later.
  • Brenton Ashworth – Web Apps in Clojure and ClojureScript with Pedestal. Good talk demonstrating and explaining Pedestal. Might have been a little bit too abstract, but I guess you can only fit so much in one hour. Included an impressive dashboard demo (great user experience and data visualizations).
  • David Pollak – Getting Pushy: Pushing data from server to browser. The creator of Lift, appropriately dressed in a lab coat, did a very good job talking about client-server (or server-client?) interaction in web apps. It involved communication in both directions, nicely encapsulated in one accessible piece that solves all the pain of possible failure modes in HTTP and network itself. Included very slick demos in AngularJS and Lift, as well as a little experiment in Clojure (Plugh) based on core.async and clang. Very good.
  • Josh Bloch, Bob Lee – Java Puzzlers, Strange Loop Edition. A handful of entertaining puzzles that often can suprise even veteran Java developers. Java specification seems to have infinite amounts of craze corners, apparently created only so Josh can have fun with the puzzlers.
  • Chris Granger – Finding a way out. The inventor of Light Table unveiled another interesting project supposed to make programming different and more accessible: Aurora. It made for some very slick and impressive demos, but it’s far from finished or usable as an end-to-end product. This time it looks like the target is not a developer of complex apps, but rather the “spreadsheet programmer”. Really nice, but not sure if very practical, and it would be great to actually see a finished, polished environment (Aurora or Light Table).
  • Douglas Hofstadter – What is a strange loop and what is it like to be one? Very good closing keynote, related to Hofstadter’s books. Impressive speech on life, perception, abstraction and consciousness.
  • David Stutz – Thrown for a Loop: A Carnival of Consciousness. Performance/theater very closely related to the preceding speech and Hofstadter’s works. Good deal of discussion of the material and immaterial, self-consciousness and imagination. Appropriately included variations of Goedel theorem, Escher graphics (plus a good deal of Emacs) and live play of Bach fugues by a brass quintet. Plus some very pleasant acrobatics. Very spectacular and unusual, I wish my English sucked less so I could get more of it.

Final Thoughts

I’m using the words “very good”, “great” and “impressive” all the time. It’s boring, but it doesn’t even do justice to the quality of talks and the whole conference. I had very high expectations for the whole thing, and I tend to be hard to please, yet the reality was simply a dream come true.

In some ways the conference was surprisingly minimalistic – I did not even get a note pad or printed program, no sponsor adverts, well nothing except for the ID badge, breakfast at the hotel and cold catered lunch. On the other hand, we got the wild party at the City Museum, the conference venue was very spectacular and comfortable, not to mention the speaker list. I do not mind the tradeoff at all! That’s the way to do it – cut the semi-important stuff to minimum and spend all the money on what really matters and becomes a truly unforgettable experience. Hats off to Alex Miller for putting it all together.

Learning to Fail

Life, Programming, Science, , ,

Back at university, when I dealt with much low-level problem solving and very basic libararies and constructs, I learned to pay attention to what can possibly go wrong. A lot. Implementing reliable, hang-proof communication over plain sockets? I remember it today, a trivial loop of “core logic” and a ton of guards around it.

Now I suspect I am not the only person who got used to all the convenient higher-level abstractions so much that he began to forget this approach. Thing is, real software is a little bit more complex, and the fact that our libraries kind of deal with most low-level problems for us doesn’t mean there are no reasons to fail.

Software

As I’m reading “Release It!” by Michael T. Nygard, I keep nodding in agreement: Been there, done this, suffered that. I’ve just started, but it’s already shown quite a few interesting examples of failure and error handling.

Michael describes a spectacular outage of an airline software. Its experienced designers expected many kinds of failures and avoided many obvious issues. There was a nice layered architecture, with proper redundancy on every level from clients and terminals, through servers, through database. All was well, yet on a routine maintenance in database the entire system just hung. It did not kill anyone, but delayed flights and serious financial losses have an impact too.

The root cause turned out to be one swallowed exception on servers talking to the database, thrown by JDBC driver when the virtual IP of the database server was remapped. If you don’t have proper handling for such situations, one such leakage can lock the entire server as all of its threads wait for the connection or for each other. Since there were no proper timeouts anywhere in the server or above, eventually everything hung.

Now it’s easy to say: It’s obvious, thou shalt not swallow exceptions, you moron, and walk on. Or is it?

The thing is, an unexpected or improperly handled error can always happen. In hardware. Or a third party component. Or core library of your programming language. Or even you or your colleague can screw up and fail to predict something. It. Just. Happens.

Real Life

Let’s take a look at two examples from real life.

Everyone gets in the car thinking: I’m an awesome driver, accidents happen but not to me. Yet somehow we are grateful for having airbags, carefully designed crumple zones, and all kinds of automatic systems that prevent or mitigate effects of accidents.

If you were offered two cars at the same cost, which would you choose? One is in pimp-my-ride style with extremely comfortable seats, sat TV, bright pink wheels and whatever unessential features. But it breaks down every so often based on its mood or the moon cycle, and would certainly kill you if you hit a hedgehog. The other is just comfortable enough, completely boring, no cool features to show off at all. But it will serve you 500,000 kilometers without a single breakdown and save your life when you hit a tree. Obvious, right?

Another example. My brother-in-law happens to be a construction manager at a pretty big power plant. He recently took me on a trip and explained some basics on how it works, and one thing really struck me.

The power station consists of a dozen separate generation units and is designed to survive all kinds of failures. I was impressed, and still am, that in power plant business it’s normal to say stuff like: If this block goes dark, this and this happens, that one takes over, whatever. No big deal. Let’s put it in a perspective. A damn complicated piece of engineering that can detect any potentially dangerous conditions, alarm, shut down and fail over just like that. From small and trivial things like variations in pressure or temperature, through conditions that could blow the whole thing up. And it is so reliable that when people talk about such conditions, rare and severe as they are, they say it in the same tone as “in case of rain the picnic will be held at Ms. Johnson’s”.

Software Again

In his “After the Disaster” post, Uncle Bob asked: “How many times per day do you put your life in the hands of an ?if? statement written by some twenty-two year old at three in the morning, while strung out on vodka and redbull?”

I wish it was a rhetorical question.

We are pressed hard to focus on adding shiny new features, as fast as possible. That’s what makes our bosses and their bosses shine and what brings money to the table. But not only them, even we (the developers) naturally take most pride in all those features and find them the most exciting part of our work.

Remember that we’re here to serve. While pumping out features is fun, remember that those people simply rely on you. Even if you don’t directly cause death or injury, your outages can still affects lives. Think more like a car or power station designer, your position is really closer to theirs than to a lone hippie who’s building a little wobbly shack for himself.

When an outage happens and also causes financial loss, you will be to blame. If that reasoning does not work, do it for yourself – pay attention now to avoid pain in future, be it regular panic calls at 3 AM or your boss yelling at you.

More Stuff

Michael T. Nygard ends that airline example with a very valuable advice. Obvious as it may seem, it feels different if you realize it and engrave it deep in your mind. Expect failure everywhere, and plan for it. Even if your tools handle some failures, they can’t do everything for you. Even if you have at least two of each thing (no single point of failure), you can still suffer from bad design. Be paranoid. Place crumple zones on every integration point with other systems, and even different components of your system, in order to prevent cracks from propagating. Optimistic monoliths fail hard.

Want something more concrete? Go read “Release It!”, it’s full of great and concrete examples. There’s a reason why it fits in a book and not in a blog post.

Why Is Science Cool?

Life, Science, ,

It can be observed that not only programming, but also science, technology and engineering, are losing their popularity. Americans even consider it a national security risk.

It may be so because today is more about consumption (or taking) rather than creation (or expressing). We prefer to take everything for granted and avoid whatever seems unpleasant. Science is pretty high on the list – it requires discipline and concentration, and that seems to be too much.

What can we do? Show that science is not about books and study of no apparent purpose. Instead, show how it affects our daily life and can actually be cool. Don’t start with dull books and formulas. Show the goal and applications, fascinate, then teach and explain.

“Run, rabbit, run.
Dig that hole, forget the sun.”

I was happy to discover the TV & radio campaign which promotes maths in Poland. It hits the sweet spot. It consists of several professional short adverts which describe how mathematical concepts apply to apparently maths-free fields such as music, art, architecture, biology etc.

For instance, they tell how the quality of music is affected by theoretical intervals, or how everything in natural environment seems to follow the golden ratio. They also involve some renown professionals who describe how maths helped them in their work.

Now, if only the teachers followed. I wish they told me that before all the formulas.