What’s with all these parentheses?

Posted on by Felix H. Dahlke

I’m fairly certain that every programmer will, at some point in his career, say something along the lines of:

I don’t know, what’s with all these parentheses?

And he will be talking about Lisp. I certainly did a few years ago. It’s quite strange when you’re used to other languages, why would anyone want all these parentheses?

Why have something like this:

(filter (lambda (price)
          (< price 10))
        prices)

When you can have it like this:

prices.select do |price|
    price < 10
end

Indeed, there is not much syntactical sugar, as Rubyists say, in most Lisp dialects. If you think about it, there isn’t really much syntax at all: You’re really just writing a syntax tree in Lisp’s list literals. Considering that, the syntax is actually pretty clear. Let’s look at what this might look like written in JavaScript’s array literals:

[filter, [lambda, [price],
           [lt, price, 10]],
         prices]

Not that much different, eh? That’s what Lisp is. And that’s why Lisp macros are so powerful: You can easily transform these data structures into something else, changing your code. I can’t think of a single other language that lets me do that. Another benefit is that when looking at Lisp code, you normally don’t have to wonder about things like operator precedence and strange literals you didn’t know, the syntax is unambiguous.

So if the parentheses are what’s separating you and Lisp, give it a chance. No one I know who gave Lisp the chance it deserves ever regretted it, whether or not it became their main language.

The Mustache way

Posted on by Felix H. Dahlke

Mustache is my favourite template engine.

It’s the only one I know that tries to keep logic out. There is a wee bit of logic, but just the bits without which it wouldn’t make sense to use it.

The traditional approach to templating looks like this:

  1. Have a model of the data relevant for a view
  2. Insert bits of that model into the view

The problem with this is that the second step is quite complicated. In most cases, you’re not rendering the model 1:1 onto the view, you convert data types, format dates, bring some bits of data together and separate others.

There are many powerful template engines that are very good at that. Some allow you to insert code, others have their own expression language. Some have mechanisms for extending the template language so you can add all the features you need.

When people used to that see Mustache, they usually want to add features, or they simply dismiss it as not being powerful enough.

Here’s the thing: This is code. Why do it in the template at all costs? Why not do this in actual code? Why not use your preferred language?

This is my approach to templating with Mustache:

  1. Have a model of the data relevant for a view
  2. Prepare it for the template (I usually copy and adjust the model)
  3. Render the prepared model 1:1 on the view
With this, you have the view logic in your code and your templates stay clean. How it ought to be, if you ask me.

JVM language popularity

Posted on by Felix H. Dahlke

I was lately interested in how popular the major JVM languages are in comparison, so I did some quick tests.

I compared Java, Scala, Groovy, Clojure and JRuby. I included both JRuby and Ruby in my queries, because JRuby isn’t really a distinct language.

The tests

Google

Quite obvious, eh? I searched for “x language” where x is one of the languages and wrote down the number of results. I’m fully aware that this isn’t a very good test.

Ruby 97,400,000
Java 46,200,000
Scala 29,200,000
Groovy 17,700,000
Clojure 3,460,000
JRuby 1,770,000
               

I thought Java would come out on top, surprised me.

Tiobe

The good old programming language popularity index.

Java #1
Ruby #13
Groovy #31
Clojure >#50
Scala >#50
JRuby Not listed
               

I thought Scala would do way better than Groovy.

GitHub

The most popular project hosting service.

Ruby #2
Java #5
Scala #18
Clojure #21
Groovy #22
JRuby Not listed
               

Scala, Clojure and Groovy are pretty close here.

StackOverflow

Probably the most important Q/A site for programmers.

Java 218,432
Ruby 41,435
Scala 8,104
Groovy 3,772
Clojure 2,762
JRuby 1,051
               

Java and Ruby are quite popular, the others less so.

Conclusion

Unsurprisingly, Java is by far the most popular language. So if alternative JVM languages are the future, the future doesn’t seem to be quite here yet.

The second place goes to Ruby. Ruby, not JRuby – it’s hard to figure out what percentage of the Ruby community is using JRuby.

Scala, Groovy and Clojure are similar in popularity. Sometimes Scala is on top, sometimes Groovy. Nonetheless, I’m actually most impressed by Clojure. It did pretty well, considering that it’s radically different to Java/Groovy/Scala and only 5 years old. (Groovy and Scala are both 9 years old, Java and Ruby both 18.)

Bottom line: When considering which JVM language (other than Java) to use, popularity can’t really be a factor. That’s good.

Clostache 1.0 – now spec compliant

Posted on by Felix H. Dahlke

I normally merely tweet about new releases of my pet projects, but Clostache, a Mustache parser for Clojure, is hitting 1.0, which I think warrants a blog post.

I changed quite a bit, in order to achieve compliance with the Mustache spec. The spec wasn’t around when I first wrote Clostache, and I learned about it’s existence just a few weeks ago from someone ranting on Twitter. Having a non-compliant Mustache parser didn’t feel right, so I changed that.

I had to fix a few bugs (mostly whitespace issues that don’t matter when producing HTML) and implement two features I never needed and thus ignored: partials and set delimiters. I also had to implement two features I didn’t know about (nor do they seem to be documented anywhere): dotted names and implicit iterators. See the README for examples. I kept the parser core mostly intact because it’s stable and mature by now, but some of the features would have been easier to implement with imperative parsing logic. It would also be faster, but Clostache has no performance issues, so that reason isn’t good enough either.

Anyway, if you’re using Clostache, please update to 1.0 and let me know if you run into any problems.

Make Emacs evaluate Clojure in 5 minutes

Posted on by Felix H. Dahlke

Emacs is my editor of choice for Clojure development (as for all Lisps), and according to the State of Clojure 2011 survey, that’s true for 68% of all Clojure developers.

Yet from what I’ve seen, some Emacs using Clojure developers don’t evaluate their code in Emacs, and the survey shows that 20% of all Clojure developers use the command-line REPL. Read on to figure out how to change that in just about 5 minutes.

“But why would I want to evaluate Clojure in Emacs?”, you might ask. Here’s why I can’t live without it: When I was new to Lisp, I missed the step-by-step debugger I knew from imperative languages. That changed when I started to write Emacs Lisp and got used to evaluating code in the editor. In functional programming, you avoid mutable state, especially global mutable state, so you can normally locate and fix bugs by evaluating suspicious parts of your program in isolation. Proper runtime inspection is still useful and necessary in some cases, and there are tools for that, but evaluating code in Emacs has been sufficient for me so far.

While most Clojurians seem to swear by SLIME to achieve this, I’m not particularly comfortable with it. It’s big, annoying to set up and there has been no release or tag to date, you have to get the latest code from CVS. Nothing you’re likely to get up and running in 5 minutes. If you’ve got some free time, do try SLIME, you might like it. If you, like me, would rather avoid it: Read on to learn how to set up inferior-lisp-mode (where inferior-lisp stands for Lisps other than Emacs Lisp) for Clojure.

Setting up inferior-lisp-mode

All you really need to do is set the variable inferior-lisp-program to the command that invokes the Clojure REPL.

Leiningen

If you use Leiningen for all projects, add the following to your init.el:

(add-hook 'clojure-mode-hook
          (lambda ()
            (setq inferior-lisp-program "lein repl")))

Maven

For Maven users, it takes a bit more effort. The inferior-lisp command will execute the configured command in the same directory as the file you are currently visiting, which is probably not the directory where your Maven pom.xml resides. Maven, however, wants you to execute most commands in that directory. To circumvent this, I wrote a wrapper script that locates the pom.xml before invoking Maven. Armed with that, you can set up inferior-lisp-program as follows:

(add-hook 'clojure-mode-hook
          (lambda ()
            (setq inferior-lisp-program "smvn clojure:repl")))

Both Leiningen and Maven

When you are working with both Leiningen and Maven projects (like I do), things get a little more complicated. I thought about writing an Emacs Lisp function that automatically discovers whether a project is a Leiningen or a Maven project and invokes the respective command, but decided to shave that yak later. Instead, I use Emacs’ dir local variables to set the inferior-lisp-program for each project. For a Leiningen project, just add a .dir-locals.el file with the following content to your project’s root directory:

((clojure-mode . ((inferior-lisp-program . "lein repl"))))

For Maven projects, do the same with the following content:

((clojure-mode . ((inferior-lisp-program . "smvn clojure:repl"))))

And to eliminate the annoying warning that pops up whenever the dir locals are set, add the following to your init.el:

(add-hook 'clojure-mode-hook
          (lambda ()
            (setq safe-local-variable-values
                  '((inferior-lisp-program . "lein repl")
                    (inferior-lisp-program . "smvn clojure:repl")))))

Using inferior-lisp-mode

Now that you’ve set up inferior-lisp-mode, you can just open a Clojure file and type: M-x inferior-lisp, which starts a Clojure REPL in the current Emacs window. You can then use the REPL conveniently from within Emacs, and evaluate s-expressions in your code by placing the cursor on closing parantheses and pressing C-x C-e.

That’s it, have fun :)

Settling down with ClojureScript

Posted on by Felix H. Dahlke

I recently blogged about using ClojureScript for a new internal application at work. After some initial difficulties, I’m finally up to speed with it. I thought it’s about time I shared some more insights.

Since the application we’re building offers a REST API to be consumed by multiple clients, we decided to eat our own dog food from the start and make the web frontend use the same API. So what we’re doing with ClojureScript is a pure client-side web project.

Generating HTML

Since we’re making a pure client-side application, we needed some kind of client-side templating. I didn’t feel like bothering with HTML after getting accustomed to Hiccup in Compojure, so we used the dom-helpers namespace from the TwitterBuzz ClojureScript demo. Now we can generate markup like this:

(dom/build [:div#content
            [:h1 title]
            [:div#messages
             [:div#message-input
              [:div
               [:label "Your name:"]
               [:input#author-name-input {:type "text"}]]
              [:textarea#message-textarea]
              [:button#send-button "Send message"]]
             [:div#message-list]]]))

The dom-helpers are basically a nice abstraction for Google Closure’s DOM manipulation utilities. I’m sure that ClojureScript will include something like that in the future.

Avoiding Google Closure

Knowing that amazing web applications like Google Docs use it, I really wanted to give Google Closure, which is included in ClojureScript, a chance. The utility functions are actually quite handy, but the lack of documentation makes using most parts, e.g. the UI widgets, a royal pain. Add to that the fact that it feels more like using a cumbersome Java framework than a JavaScript library, I decided to avoid it where possible. Google Closure is technically sophisticated, but lacks elegance and convenience.

Building with Leiningen

When we started to use ClojureScript, we built it from source and invoked the ClojureScript compiler from a Makefile. I can’t begin to tell you how weird this felt. Since we’re using Leiningen to build the backend, we decided to use it for the frontend as well, so I wrote a custom Leiningen plugin that performs the tasks we need, e.g. calling the ClojureScript compiler and packing the output into a WAR file. We’re not planning to release this plugin, it’s pretty hard wired towards our needs and not really thought through.

Instead, I started to contribute to lein-clojurescript, first making it work with a stable release of Leiningen. The author likes my ideas about offering multiple modes for development and production, and making the respective options configurable in the project.clj, so that’s the plugin you want to use.

Promising stuff

REPL in the browser

When ClojureScript was first announced a while ago, the REPL used Rhino for JavaScript evaluation and DOM manipulation was not possible. The ClojureScript developers have addressed that problem and made it possible to use the Browser as evaluation environment. It looks promising, but I didn’t really find the time to play around with it yet.

Compiling automatically

Compiling without optimizations is pretty fast, but manually invoking the compiler before refreshing the browser still feels wrong when doing web development. cljs-watch attempts to solve that problem by watching your ClojureScript sources, recompiling as soon as anything changes, e.g. you save a file. I trie it, but couldn’t make it work right away, so I postponed that. I guess something similar can be build into lein-clojurescript.

Problems

As I told you at the beginning of this post, I’m pretty happy with ClojureScript by now. We had to put some extra energy into setting up our development environment and build system, but now we’re up to speed. A few problems, however, remain.

No unit testing

I haven’t yet invested the time to figure out how to execute unit tests for ClojureScript code, it will probably be easier now that we build with Leiningen. This is something I’ll have to approach soon, working without tests feels really bad.

Still no release

I really don’t know why the Clojure/core folks announced ClojureScript without having a release candidate of ClojureScript ready, or even a tag. That hasn’t changed. At first, we forked the ClojureScript project on GitHub and created a tag to ensure reproducible builds. Now that we use lein-clojurescript, we use the ClojureScript version from Clojars, but that doesn’t feel much better – it’s still an arbitrary snapshot.

Conclusion

Despite the problems, ClojureScript made a very good first and second impression. It proved very stable (I didn’t come across a single compiler weirdness, of which I’ve seen plenty in GWT), which made me take languages that compile to JavaScript seriously again. Instead of whining about programmer inconvenience and immature tooling, I’ve decided to contribute, starting with lein-clojurescript. Once that’s powerful enough to replace our custom plugin, I’ll see if I can help out with ClojureScript itself, it really needs an initial release.

Back to game development

Posted on by Felix H. Dahlke

After busying myself with other things for a couple of months, I feel a strong urge to return to game development. I haven’t blogged about this, but my wife and I work on a few games in our spare time. The one that’s most likely to ship this year is a puzzle game for Android, and I’m determined to make that happen.

I’ve been a little less motivated to work on that particular game because, well, it’s a puzzle game. The code is pretty much final, but most of the game content is missing; there are only a few boring levels with ugly programmer art at this point. Designing levels for a puzzle game on pen and paper doesn’t really work for me, so I began to write a level editor. A major feature will be the possibility to create random levels, so that I will hopefully be able to generate a level, attempt to solve it and rate its difficulty.

I really enjoy Android development, but one of the things that keeps annoying me is that you’re stuck with Java. I love Java as a platform, but I never felt much love for the Java language, far too limited for my taste. Fortunately, I can use Clojure for the level editor. It is possible to write Android apps in Clojure and other JVM languages, but there is apparently a performance penalty involved, and I can’t have that. I guess I’ll use C++ for our next Android game, despite the fact that I won’t be able to use the pretty neat Android framework.

Anyway, stay tuned.

Adventures with ClojureScript

Posted on by Felix H. Dahlke

Since Clojure is one of my favourite programming languages, I took a chance and used the brand new ClojureScript on a small project.

I was sceptical when ClojureScript first came out: Having used Google Web Toolkit for years has made me a bit nervous about compiling other languages to JavaScript. Such a layer of indirection can seriously complicate things, and make development feel disturbingly like magic.

On the other hand, I thought that Clojure, being a Lisp dialect, would translate much more nicely into JavaScript – which is very similar to Lisp - than Java. So I gave it a go.

Working with ClojureScript

ClojureScript is very young, so there is hardly any tool support at this point. Neither Maven nor Leiningen can build ClojureScript projects, so I wrote a good old Makefile.

Compiling felt weird, because the usual JavaScript workflow is to just refresh the page. With ClojureScript, I had to invoke make first, which is fortunately pretty fast, at least if you don’t have much code.

I typically spend a lot of time in the browser’s JavaScript shell, so I expected to use the ClojureScript REPL extensively. It compiles ClojureScript into JavaScript on-the-fly and interpretes that with Rhino, a JavaScript engine for the JVM. That doesn’t only sound weird, it’s also a bit unstable. I had mysterious exceptions all the time, and at some point it even crashed my machine. I didn’t investigate these problems further, because I really had to get stuff done. What I really want – DOM manipulation – is apparently not possible in the ClojureScript REPL anyway, and the normal Clojure REPL was sufficient for everything else.

As for debugging in the browser, I had to break that habit as well: The JavaScript generated by ClojureScript was hardly readable, even with optimisations turned off and pretty printing.

These are quite a few limitations, and as expected, they did slow me down. Then again, being able to use Clojure instead of JavaScript made up for that a bit (I really like JavaScript, but I feel that I’m more productive with Clojure).

Functions and properties

ClojureScript worked like I expected it to most of the time, but there was one particular weirdness that made me blink: If you call a JavaScript function, you will get the function’s source code as a return value, not the function’s return value. It does make sense from a technical point of view, but I’m pretty sure it will make every single aspiring ClojureScript programmer curse at some point.

This is how you would normally execute a function of an object:

(.someFunction someObject)

And this is how you execute a function and get it’s return value in ClojureScript:

(. someObject (someFunction))

I couldn’t find any documentation on working with properties in ClojureScript, so I’ll share what I figured out:

Get a property:

(.someProperty someObject)

Set a property:

(set! (.someProperty someObject) newValue)

Working with Google Closure

ClojureScript comes with the Google Closure Library, a large collection of JavaScript utilities and widgets, created by Google and used on amazing web applications like GMail and Google+. I had been curious about Closure, so I decided to use it for this project.

My first impression is that it is less elegant and concise than jQuery and other JavaScript libraries I’ve used. I had to write an annoying amount of boiler plate code to create a simple UI programatically. In Clojure, where things are usually very elegant and concise, this felt even weirder – not too different from using Swing in Clojure on the JVM.

To make things worse, the documentation isn’t up to Google standards, but I guess that would change as the library becomes more popular. If it does.

Conclusion

ClojureScript is interesting, but apparently not intended for production use yet – there hasn’t even been a release. It needs to mature, providing a stable REPL and a nice functional layer above the Google Closure Library (or some other library, or something written in ClojureScript).

With languages that compile to JavaScript becoming more common and with the prospect of debugging support, ClojureScript is definitely something I will keep an eye on.

Contributing to Chromium

Posted on by Felix H. Dahlke

I always wanted to work on a web browser. Probably because I’m both a front-end focused web developer and a low-level loving C++ programmer, and web browsers are among the few types of software that require such a combination.

Moreover, I recently felt the urge to lay hands on large, established code bases. I believe that working with existing code will take me further in my quest of becoming a better programmer than starting yet another small project. So I decided to contribute to Chromium, the open source project behind Google Chrome.

Why Chromium?

Well, mostly because it’s my favourite browser, and because it is in my opinion the one that pushes the web the most right now. I also think that a few more non-Googlers working on it would be good for the project. Not that I distrust Google on this, but I believe that an open source project, especially such an important one, needs contributors from all over the industry. A too dominant player can, even with good intentions, harm a project in the long run: Remember what has happened after Oracle bought Sun.

Another reason why I picked Chromium was that I thought it was a small, modern, lightweight piece of software, unlike Firefox, whose code base appears to be an ancient monster. Guess what? Chromium’s code base is a monster too, at least in terms of size. Including dependencies, we’re talking about several gigabyte of source code here.

The code

Nonetheless, the code quality looks good to me. Google’s (partially weird) code style is followed consistently and the code I’ve seen so far looked clean and polished. I don’t understand the complete architecture yet, but what I’ve seen and read seems promising. I have only two problems with the code:

Monolithic build

I found it difficult to work on just one area, in my case the UI. You have to statically link all of Chromium (which takes about 10 minutes on my, admittedly slow, netbook) in order to see even a small change. I would prefer it if there was some way to fiddle with design and layout with a faster turnaround. Some people on the #chromium IRC channel seem to be unhappy with this as well.

Redundant UI code

Considerable parts of the UI code are effectively duplicated for each supported platform: Windows, Mac and Linux. The Windows code is written with WinAPI and WTL, the Mac code with Cocoa and Objective-C and the Linux code with GTK. I’m sure they have their reasons, one of which is very likely performance, but I still think there could be more code reuse among the platforms. I have seen essentially the same code on different platforms, written in a slightly different way, probably by a different person. As is the problem with code duplication like this, some bugs get fixed only in some versions of the code. The status bubble (the thing that appears if you move the mouse cursor over a link) is a good example of this: It slides perfectly on Windows, flickers a bit on Mac and disappears completely on Linux.

Contributing

In order to find something to contribute, I had a look at the Chromium issues marked as GoodFirstBug and picked one that concerned the sliding of the status bubble (see above) on Linux. After identifying the problem and fooling around with the code, I noticed that this was a non-trivial GTK problem, the kind of thing I’d like to avoid with my first patch, not having used GTK before. So I picked an issue with the downloads tab which was all in all about one hour of work, naturally not counting compilation time. To my delight, the download tab’s UI was written in HTML/JS, which made the UI work a no-brainer.

It was fun to work with the code, but the compile times were pretty annoying. My desktop box at home is quite fast, but I rarely use it ever since I became a dad. So I worked on the train, which means on my netbook. I usually don’t have any problems programming on it – even Eclipse is usable – but Chromium development was tough. A trivial change meant that I would have to wait for 10-15 minutes. A git pull meant that I had to compile over night. It was mind-numbing.

Was that worth it? Yeah. Sure, my issue wasn’t a big deal, but now I can tackle more difficult ones, maybe even become a committer if I stay motivated. That would allow me to shape the future of the web, which is a considerable level-up for a web developer. Not sure if I can keep my motivation when having to work on that slow netbook though. Maybe I can use some of the 20% time at work for this, I guess Chromium development would be much more enjoyable on a powerful iMac.

Update

My company being awesome, I was indeed able to hack on Chrome during 20% time. I implemented Kiosk mode for Mac. As expected, it was more enjoyable with the iMac. I have a lot of 20% projects going on, but I’m definitely planning to contribute more in the future.

Switch to our mobile site