Posts Tagged ‘Clojure’

« Older Entries

Connection Management in MongoDB and CongoMongo

Saturday, February 4th, 2012

I decided to take the opportunity offered by Jacek Laskowski (in Polish) and take a closer look at interaction with MongoDB in Clojure. It has a nice, challenging learning curve as I haven’t done much practical work in Clojure and I’ve never actually dealt with Mongo before. Double win – learning two interesting things at a time.

The obvious choice for the integration is CongoMongo. It’s really easy to get it all set up and working. The official docs encourage you to simply do this:

(def conn (make-connection "mydb")
 
(set-connection! conn)
 
(insert! :robots {:name "robby"})
 
(fetch-one :robots)
 
; ... and so on

Easy. Too easy and comfortable. Coming from the old good and heavy JDBC/SQL I felt uneasy with the connection management. How does it work? Does it just open a connection and leave it dangling in the air the whole time? Might be good for a quick spike in REPL, but not for a real application which needs concurrency, is supposed to be running for days and weeks, and so on. How do you maintain it properly?

clojure.contrib.sql has with-connection. That opens the connection, runs something with it and then eventually closes it. CongoMongo has with-mongo, but all it does is bind the argument to *mongo-config* and execute body. Nothing is ever opened or closed.

That seemed insane and broken, until I took a step back and compared source of CongoMongo to documentation of underlying Java driver for MongoDB. The light dawned.

What make-connection really does is create an instance of Mongo and DB (if database name was provided). The result of this function is plain map: {:mongo #<Mongo>, :db #<DB>}.

Javadoc for Mongo say it’s a database connection with internal pooling. For most application, you should have 1 Mongo instance for the entire JVM. A page dedicated to Java Driver Concurrency explains it in more detail: The Mongo object maintains an internal pool of connections to the database. For every request to the DB (find, insert, etc) the java thread will obtain a connection from the pool, execute the operation, and release the connection..

At first I thought CongoMongo docs were misleading. The truth is, it’s just a wrapper for the Java driver. It’s fair for it to assume you know the basic principles of the underlying driver.

So what is called a “connection” here (the Mongo class) is in fact a much more sophisticated object. It maintains a connection pool and creates nice little DB objects for all the data handling, which in turn are smart enough to maintain the actual low-level connections for you. No ceremony, just gets out of the way as quickly as possible and lets you get the job done.

This is amazingly simple and elegant compared to JDBC / JEE / SQL. I guess I soon will be scratching my head over ACID, but at the moment I’m pleasantly surprised with the look of things.

Tags: ,
Posted in Programming | No Comments »

Fill and Print an Array in Clojure

Thursday, October 27th, 2011

In a post in the “Extreme OO in Practice” series (in Polish), KozioĊ‚ek used the following example: Fill a 3x3x3 matrix with subsequent natural numbers and print it to stdout.

The example in Java is 27 lines of code (if you remove comments). It is later refactored into 55 lines that are supposed to be more readable, but personally I can’t help but gnash my teeth over it.

Anyway, shortly after reading that example, I thought of what it would look like in Clojure. Here it is:

(def array (take 3 (partition 3 (partition 3 (iterate inc 1)))))
(doall (map #(doall (map println %)) array))

That’s it. I know it’s unfair to compare this to Java and say that it’s shorter than your class and main() declaration. But it is, and it makes the full-time Java programmer in me weep. Anyway, I have two observations.

As I like to point out, Java is awfully verbose and unexpressive. Part of the story is strong typing and the nature of imperative programming. But then again, sometimes imperative programming is very clumsy.

Secondly, this solution in Clojure presents a completely different approach to solving the problem. In Java you would declare an array and then iterate over it in nested loops, incrementing the “current value” in every innermost iteration. In Clojure, you start with an infinite sequence of lazy numbers, partition (group) it in blocks (that’s one row), then group those blocks into a 2D array, and take 3 of those 2D blocks as the final 3D matrix. Same thing with printing the array. You don’t iterate over individual cells, but naturally invoke a function on a sequence.

The difference is subtle, but very important. Note how the Java code needed 4 variables for the filling (“current value” and 3 indexes) and 3 indexes for printing. There was a very clear distinction between data and what was happening to it. In Clojure you don’t need to bother with such low-level details. Code is data. Higher-order functions naturally “serve as” data in the first line, and are used to iterate over this data in the second.

Tags: , ,
Posted in Programming | No Comments »

Java Does Not Need Closures. Not at All.

Monday, October 17th, 2011

I hope that most Java developers are aware of the Swing threading policy. It makes sense, except for that it’s so insanely hard to follow. And if you do follow it, I dare say there is no place with more ridiculous boilerplate.

Here’s two examples that recently got me scratching my head, gnashing my teeth, and eventually weep in disdain.

Case One: Display Download Progress

class Downloader {
    download() {
        progress.startDownload();
        while(hasMoreChunks()) {
            downloadChunk();
            progress.downloadedBytes(n);
        }
        progress.finishDownload();
    }
}
 
class ProgressDisplay extends JPanel {
    JLabel label;
    startDownload() {
        SwingUtilities.invokeLater(new Runnable() {
            public void run() {
                label.setText("Download started");
            }
        });
    }
    downloadedBytes(int n) {
        SwingUtilities.invokeLater(new Runnable() {
            public void run() {
                label.setText("Downloaded bytes: " + n);
            }
        });
    }
    finishDownload() {
        SwingUtilities.invokeLater(new Runnable() {
            public void run() {
                label.setText("Download finished");
            }
        });
    }
}

Solution? Easy! Just write your own JLabel wrapper to hide this mess. Then a lengthy JTable utility. Then a JProgressBar wrapper. Then…

Case Two: Persist Component State

Task: Save JTable state to disk. You’ve got to read table state in EDT, but writing to disk from EDT is not the the best idea.

Solution:

ExecutorService executor = Executors.newSingleThreadExecutor();
 
void saveState(final Table table) {
    SwingUtilities.invokeLater(new Runnable() {
        public void run() {
            TableColumns state = dumpState(table);
            executor.execute(new Runnable() {
                public void run() {
                    saveToDisk(table.getTableKey(), state);
                }
            });
        }
    });
}

Two inner classes, inheritance, all those publics and voids and parentheses. And there is no way to avoid that!

In a sane language, it could look like:

(defn save-state [table]
  (do-swing
    (let [state (dump-state table)]
      (future (save-to-disk table state)))))

All semantics. Little ceremony, almost no syntax. If you counted characters in both solutions, I guess the entire second snippet is as long as the first one somewhere to the invocation of invokeLater. Before you get to the actual logic. No strings attached, do-swing and future are provided by the language or “official” utils.

Bottom Line

I recently saw two presentations by Venkat Subramaniam on polyglot programming where he mercilessly made fun of Java. In his (vaguely remembered) words, when you write such code for the whole day, you come back home and weep. You can’t sleep because you’re tormented by the thought of the abomination you created, and that you will have to deal with tomorrow. If your children asked you what do you for living, would like them to see this?

Tags: , , ,
Posted in Programming | No Comments »

Yaclot 0.1.2: Extended Date Conversions

Saturday, October 8th, 2011

I released a new version of my Clojure conversion and map transformation library to Clojars. It includes two new features:

1. Added natural conversion between Long and Date.

2. You can pass a collection of formats for converting String to Date. It attempts parsing with each of the formats and returns first result which didn’t throw ParseException.

(convert
  "2/12/11"
  (using-format ["yyyy-MM-dd" "M/dd/yy"]
    (to-type java.util.Date)))
; => #<Date Sat Feb 12 00:00:00 CET 2011>

Enjoy!

Tags: ,
Posted in Programming | No Comments »

“Processing in Clojure” Made Functional

Friday, March 18th, 2011

Invited to nitpicking by Jacek Laskowski in his latest post titled Processing in Clojure, I decided to write my own version of his flower-drawing application. I find it an interesting challenge and illustration for my previous post about functional thinking and the use of language and decomposition.

Full source code is below, but let’s start with what I did not like and decided to rewrite.

I find Jacek’s code rather structural and hard to comprehend. Everything happens in one big function and it takes quite a while to understand what it does and how. It’s hard to see why angle is an angle, and variables called a and value don’t make it any easier.

Thus my first two improvements are: Break it down into smaller pieces and use more informative names. For instance, value is in fact scale. I extracted drawing flower to a separate function with clear distinction to drawing petals and the central piece. I also paid more attention to variable names and hid the more complex concepts behind simple, well-named functions.

The second type of improvements is use of functional features. After a while of careful inspection I observed that angle is a simple linear sequence, and scale is result of a simple function on this sequence. I decided to replace it with a dedicated infinite sequence.

In the end, this code is longer and has more levels of abstraction, but I think it’s more functional and comprehensible. Functions are much shorter. Details no longer obscure the view, but are hidden behind more descriptive names. Now you can actually see that the applet’s draw draws a flower which consists of petals and the central piece. The central piece is a simple circle, while petals are several circles of random size and color around the center. Then only if you want you can delve into details of how their color and size are generated.

Like I mentioned in the first paragraph, I find it a nice illustration of what Paul Graham describes as building a language for the solution.

Tags:
Posted in Programming | 1 Comment »

Yaclot 0.1.0 Released (Clojure Conversion and Record Transformation Library)

Tuesday, March 8th, 2011

I released Yaclot 0.1.0 to Clojars. This version can be used to convert single values as well as records.

Converting individual values looks like:

(convert "2011-02-12" (to-type java.util.Date))
; => #<Date Sat Feb 12 00:00:00 CET 2011>
 
(convert "2/12/11"
    (using-format "M/dd/yy" (to-type java.util.Date)))
; => #<Date Sat Feb 12 00:00:00 CET 2011>
 
(convert 5000.42 (to-type String (using-format "%,.2f")))
; => "5,000.42"

Order of using-format and to-type doesn’t matter. using-format is optional (default for dates is yyyy-MM-dd).

You can also use it to convert records in a single operation:

(map-convert 
  {:dt "2011-02-12" :int 42 :label "Label"}    ; Record
  {:dt  (to-type java.util.Date)               ; Desired types
   :int (to-type String)})
; => {:dt #<Date Sat Feb 12 00:00:00 CET 2011>, :int "42", :label "Label"}

The project originated from a pet web application, where I needed to deal with parsing and formatting numbers and dates between the “presentation”, “business logic” and “database” layers.

To learn more about motivation, features and syntax, take a look at the draft in my previous post.

This version supports conversions between String, Date and numeric types. Next on the road map is exception-free error handling with local bindings and validation.

See also:

Tags: ,
Posted in Programming | No Comments »

Designing Yaclot: Generic Clojure Conversion Library

Tuesday, February 22nd, 2011

In real Clojure applications you often cannot avoid explicit type conversions, even though the language does not require you to explicitly specify types all the time. Another great feature of Clojure is its universal data structure: most of the time you don’t need anything more than a simple map or record. However, these records often need to be a bit different in different areas of the applications.

Rationale

Examples? You may have a java.sql.Date in database and backend, but your web front end provides you with a String. Or your backend uses rational numbers, but you need to format them into nice Strings for presentation.

Sometimes you operate on whole records. From database you may pull the following:

(def db-sample
  {:date    2011-02-01 ; java.sql.Date
   :balance 2042.00    ; java.math.BigDecimal
   :credit  1000.00    ; java.math.BigDecimal
   :roi     2.13 })    ; java.math.BigDecimal

… for display you would like to use:

(def presentation-sample
  {:date    "Feb 1, 2011" ; String
   :balance "$2,042.00"   ; String
   :credit  "$1,000.00"   ; String
   :roi     "2.13%" })    ; String

… and you need to support input from a web form as:

(def form-sample
  {:date    "2011-02-01"  ; String
   :balance "2042"        ; String
   :credit  "1000"        ; String
   :roi     nil })        ; Rational calculated 
                          ; by back-end from other fields

All three have very similar structure, but implementing all these transformations can be pain.

Planned API for generic converter

I am implementing a library that will do it for you, with API similar to the following:

(def db-fmt
  {:date    (to-type java.sql.Date)
   :balance (to-type java.math.BigDecimal)
   :credit  (to-type java.math.BigDecimal)
   :roi     (to-type java.math.BigDecimal) })
 
(def presentation-fmt
  {:date    (using-format "MMM d, yyyy" (to-type String))
   :balance (using-format "$%.2f"       (to-type String))
   :credit  (using-format "$%.2f"       (to-type String))
   :roi     (using-format "%.2f%%"      (to-type String)) })
 
(def form-fmt
  {:date    (using-format "yyyy-M-d" (to-type java.sql.Date))
   :balance                          (to-type java.math.BigDecimal)
   :credit                           (to-type java.math.BigDecimal) })
 
(map-convert db-sample presentation-fmt)
=> ; (similar to presentation-sample)
 
(map-convert form-sample db-fmt)
=> ; (similar to db-sample)

Another feature is generic conversion function for individual values:

(convert "2011-02-12" (to-type java.util.Date))
; => #<Date Sat Feb 12 00:00:00 CET 2011>

(convert (java.util.Date. 111 1 12) (to-type String))
; => "2011-02-12"

(convert 42 (to-type String))
; => "42"

(convert "2/12/11" (using-format "M/dd/yy" (to-type java.util.Date)))
; => #<Date Sat Feb 12 00:00:00 CET 2011>

Ideas for the Future

In the future, this library could support pre- and post-conversion validation. For instance, check that balance is not empty and date is in the correct format before conversion, and validate that balance and credit are positive once they are numbers.

Another idea is using it as a base for yet another HTML form manipulation library. I found existing libraries somewhat disappointing as they imposed to many restrictions on me. I would like to have the ability to manipulate and lay out forms as I please, and only use the bits of the library that I need right now.

Current Status

Currently much of the API and features are designed, but only the above presented part is implemented (not even supporting formats for numbers, only for dates). Once I implement conversions between all popular/basic types I will mark it 0.1 and push to Clojars. Error handling and validation are next on the road map.

The code is available at github.

Feel free to share any comments or ideas.

Tags: ,
Posted in Programming | No Comments »

TDD in Clojure: Mocking & Stubbing

Saturday, February 12th, 2011

A few minutes into my first real TDD trip in Clojure I discovered there is two reasonable ways I can do tests: Mocking or black-box testing. I decided to go for mocking, and so I discovered clojure.contrib.mock. I found the docs fairly confusing, but finally understood it with a little help of this article and research.

Verify Calls to a Function

Assuming we have a function to compute square of a number, we want to write another function for square of sum.

(defn square [x] (* x x))

Our test can look like this:

(ns squirrel.test.core
  (:use [clojure.test])
  (:use [clojure.contrib.mock]))
 
(deftest test-square-of-sum
  (expect [square (has-args [3])]
    (square-of-sum 2 1)))

This use of expect asserts that when we execute the inner form (square-of-sum 2 1), it calls square with argument equal 3. However, it does not execute square itself. The only thing that this test checks is whether square got called. In particular, it does not check what (square-of-sum 2 1) returns. We’ll get back to stubbing in a moment.

Stubbing

Let’s modify our test to also assert the final result:

(deftest test-square-of-sum
  (expect [square (has-args [3])]
    (is (= 9 (square-of-sum 2 1))))

When the test runs, it fails because square-of-sum returns nil. The reason is that expect replaces square with a stub which by default doesn’t return anything.

To have stub return a concrete value, we can use the returns function:

(deftest test-square-of-sum
  (expect [square (returns 9 (has-args [3]))]
    (is (= 9 (square-of-sum 2 1))))

Voila. Now the test passes.

To recap, what this two-line test does is:

  • Create a stub for square which returns 9 for argument of 3.
  • Assert this stub is called with argument 3.
  • Assert that square-of-sum calls this stub with argument 3.
  • Assert that square-of-sum eventually returns the correct value.

Quite a lot for such a tiny test.

Expectation Hash

You may be wondering what exactly the second argument in each binding pair is. Clojure docs call it expectation hash.

Each function that operates on an expectation hash, such as has-args or returns, has two overloaded versions. One of them only takes a value or predicate and returns a new expectation hash. Examples include (returns 9) or (has-args [3]).

The other version takes an expectation hash as the second argument. These versions are used to pass the expectation hash through a chain of decorators. Order of decoration does not matter, so (returns 9 (has-args [3])) is effectively the same as (has-args [3] (returns 9)).

Tags: ,
Posted in Programming | 1 Comment »

Learn-Clojure.com: Clojure basics distilled

Saturday, December 11th, 2010

Kyle Cordes has put up a new site dedicated to Clojure: learn-clojure.com. As you can read in it’s FAQ, it is an attempt to consolidate just the most useful introductory material in one place, presented in a way that aims to help new or prospective Clojure developers.

At this moment that means basic information on what Clojure is, as well as many useful references to tutorials, videos and books. Clearly it’s not meant to compete with clojure.org or clojuredocs.org. Nor is it another tutorial or a garbled list of everything and nothing. It’s merely a clean, concise, nicely-organized gateway to the Clojure world.

This is very important, because while Clojure is rapidly gaining popularity (some are dubbing it “the next big thing” after OOP), it may be pretty discouraging at the beginning. In this world of enthusiastic magicians, crude immature frameworks and sparse documentation, by no means has finding my way been easy. Let alone answering the fundamental questions of “why?”, “what?” and “how?” That was an empty niche, and it’s great that Kyle made an effort and organized this together.

Unfortunately, apart from the Community section (not much going on in Poland) it does not provide much information to people who already are familiar with the basics. I’d love to see such a nice list on more concrete subjects like: “What are my options for building a web app?”, “How can I access the database?”, “How do I integrate it with JEE?” As well as a showcase of successful, inspiring (and mature) applications like Incanter or Ring. But that is yet to come, or may even be out of scope of this site.

Tags:
Posted in Programming | No Comments »

The Joy of Clojure: Learning “The Clojure Way”

Wednesday, September 15th, 2010

The Joy of Clojure (cover)

Very often the journey with Clojure starts like this. You notice its growing popularity or want to attach a new tool to your belt. So you go through some quick introduction, learn the basics, get a few programs running… And eventually ask yourself: Fine, but what’s next? What’s the difference? How does one really work with Clojure?

That’s where The Joy of Clojure by Michael Fogus and Chris Houser enters the play. The goal of this book is to provide answers to these fundamental questions and teach you “The Clojure Way”.

It starts with an explanation of what Clojure is and the problems that it solves. Then it provides an overview of the language: data types, functions, collections, destructuring, composite data types and lazy evaluation. Finally it explores more advanced concepts: functional programming, metaprogramming, performance, mutation, concurrency and parallel programming.

However, by no means is it an average reference / language overview book. What it really does is explain the philosophy of LISP, functional programming and last but not least Clojure, and only then discuss all the technical stuff.

Just because the primary focus is not syntax and low-level technical details does not mean they are not explained in depth. Nothing is missing: syntax, performance, intricate details of data types and other constructs – they’re all here.

That’s where the book really shines. It’s easy to write a manual that describes a technology, but fails to describe its purpose. “The Joy of Clojure” managed to explain what Clojure really is, with all its fundamentals and idioms.

The major downside of the book is that some of the advanced low level technical / syntactical aspects are explained in a rather steep, if not discouraging way. Take macros for example. The concept is clear, but the examples could be less complicated and it falls short of explaining the basic syntax.

If you’re looking for an easy basic introduction to Clojure, or a practical guidebook on libraries, developing web apps etc. then probably this is not the best choice. But if you are up for a challenge or know some Clojure and are asking yourself the fundamental questions from the first paragraph of this review, you are going to love it. Sometimes it is steep and demanding, it may require you to reread some of the chapters later, but it definitely is worth it.

Note: This review is based on an early access edition provided by Manning Publications Co. The estimated publishing date is December 2010, but you can get a pre-release PDF already from Manning Publications.

Tags: ,
Posted in Programming | No Comments »

« Older Entries