Packaging binaries for Linux

I had the pleasure to prepare a binary (including dependencies) for Linux a few months ago. It’s been a while since I had to do that, and I forgot how difficult it is. Here’s some advice:

Compile libraries yourself

This is probably the most important point. You can either link statically or link dynamically and ship your shared objects, the latter usually being a lot less problematic. If you ship the shared objects prepared by your distribution, you will introduce lots of unnecessary direct and transitive dependencies, forcing you to add tons of additional libraries. For instance, here goes an ldd of the core SDL library on Ubuntu 10.10:

linux-vdso.so.1 =>  (0x00007fff64bce000)
libm.so.6 => /lib/libm.so.6 (0x00007f6888cfe000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f6888afa000)
libpulse-simple.so.0 => /usr/lib/libpulse-simple.so.0 (0x00007f68888f5000)
libpulse.so.0 => /usr/lib/libpulse.so.0 (0x00007f68886b4000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f6888497000)
libc.so.6 => /lib/libc.so.6 (0x00007f6888113000)
/lib64/ld-linux-x86-64.so.2 (0x00007f6889236000)
libpulsecommon-0.9.21.so => /usr/lib/libpulsecommon-0.9.21.so (0x00007f6887ec7000)
librt.so.1 => /lib/librt.so.1 (0x00007f6887cbf000)
libX11-xcb.so.1 => /usr/lib/libX11-xcb.so.1 (0x00007f6887abc000)
libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f6887786000)
libICE.so.6 => /usr/lib/libICE.so.6 (0x00007f688756b000)
libSM.so.6 => /usr/lib/libSM.so.6 (0x00007f6887361000)
libXtst.so.6 => /usr/lib/libXtst.so.6 (0x00007f688715a000)
libxcb-atom.so.1 => /usr/lib/libxcb-atom.so.1 (0x00007f6886f55000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007f6886d37000)
libwrap.so.0 => /lib/libwrap.so.0 (0x00007f6886b2c000)
libsndfile.so.1 => /usr/lib/libsndfile.so.1 (0x00007f68868c7000)
libdbus-1.so.3 => /lib/libdbus-1.so.3 (0x00007f6886684000)
libuuid.so.1 => /lib/libuuid.so.1 (0x00007f688647f000)
libXext.so.6 => /usr/lib/libXext.so.6 (0x00007f688626c000)
libXi.so.6 => /usr/lib/libXi.so.6 (0x00007f688605c000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x00007f6885e59000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007f6885c52000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00007f6885a38000)
libFLAC.so.8 => /usr/lib/libFLAC.so.8 (0x00007f68857ee000)
libvorbisenc.so.2 => /usr/lib/libvorbisenc.so.2 (0x00007f688531e000)
libvorbis.so.0 => /usr/lib/libvorbis.so.0 (0x00007f68850f2000)
libogg.so.0 => /usr/lib/libogg.so.0 (0x00007f6884eeb000)

That’s quite a lot of libraries, some of which you can’t even reasonably ship (I’m looking at you, pulseaudio). The solution? Build your required libraries yourself, and do it on the oldest and humblest distribution you want your software to work on. I’m building our current C++ game project and its dependencies on Debian Lenny, this makes it work on all reasonably current popular distributions. Furthermore, if you build the libraries yourself, you can usually configure it to compile only the features you need, reducing size and dependencies even further. This is an ldd of the core SDL library I built:

linux-gate.so.1 =>  (0xf77a7000)
libm.so.6 => /lib32/libm.so.6 (0xf76d3000)
libdl.so.2 => /lib32/libdl.so.2 (0xf76cf000)
libpthread.so.0 => /lib32/libpthread.so.0 (0xf76b5000)
libc.so.6 => /lib32/libc.so.6 (0xf755a000)
/lib/ld-linux.so.2 (0xf77a8000)

See the difference? Also make sure to ship a 32 bit version, preferably both 32 and 64 bit.

Check distribution compatibility

There are lots of Linux distributions out there, and you’ll probably want to support as many as possible. If you compile libraries yourself, you’re definitely on the right track, but you should also check for compatibility issues and see what you can do about them. You can either get dozens of distributions and check if it works, or you can use the excellent Linux Application Checker tool:

Reduce dependencies

Reducing your dependencies is good, but do you know what’s even better? Get rid of them. You really shouldn’t overdo this, but look at each of your dependencies and think about whether you really need them. Should you include a large library if you need just one function? Is there a more coherent alternative? Would it be possible and perhaps faster or easier if you just wrote the required functionality yourself?

TimTim hits the Android Market

In the past few weeks, I’ve been working on my first Android app: A kitchen timer named TimTim. It’s not exactly rocket science, but I was looking for something simple to explore the Android platform and see how the Market works. I also wanted to push this all the way through to the polish phase quickly, a state my more ambitious projects hardly ever reach.

So what I’ve uploaded a few days ago is a pretty polished kitchen timer, with a lovely fruits design done by my wife. Here’s a screenshot:

Most of the polish work went into the algorithm for rotating the dial based on finger movement. This was significantly more difficult than expected, but I think it works really well now. I also made it possible to rotate the dial with the D-Pad, in case the finger rotation doesn’t work for anyone.

Apart from that, I’ve also added the possibility to manage presets for common kitchen tasks:

Naturally, the alarm is configurable, too:

Because my wife prepared a few designs and we liked them all, I added the possibility to switch skins. Here’s the lemon skin:

And this is my personal favourite, citron:

You can get TimTim for 50 cents on the Android Market. I’m also planning to add it to a few other stores, like SlideME. We’ve already started to work on our next app, a game. Stay tuned.

Update

TimTim is now also available on SlideME and AndroidPIT.

Update 2

I’ve created a free version of TimTim, it’s on the Android Market.

Regular expressions

Earlier this month, I expressed my astonishment about the fact that the majority of software developers I’ve worked with in the last seven years doesn’t know the first thing about regular expressions:

fhd%3A It's amazing how many actual developers don't know regular expressions. It's like carpenters who don't know about hammers.

As you might have guessed, I regard regular expressions as a fundamental element of every programmer’s toolbox. However, I’m not good with metaphors, and I don’t know the first thing about carpentry, so hammer missed the point. Thomas Ferris Nicolaisen found a better analogy:

tfnico%3A %40fhd I would rather say it's like carpenters who don't know circle-saws %3B)

He’s right: Regular expressions are a specialised way to work with text, mostly relevant to programmers – not everyone who works with text in general.

Most of the other replies I got indicated that although they knew (or had once known) regular expressions, they rarely used it nowadays. I think that’s a shame, so I decided to share what I do with regular expressions on a daily basis, maybe you’ll find it useful. I do use them in code occasionally, but what I do all the time, whether I use an editor or an IDE, is searching and replacing. If you’re not at all familiar with regular expressions, I suggest this reference to make sense of the remainder of this post.

Searching

I sometimes mention that I grew up with UNIX and that’s true. One of the first things I learned about programming was how to use the tools of the Linux command-line, like grep, which is a command that allows you to search the contents of one or more files with a regular expression.

I can’t come up with a convincing example because I mostly use regular expression searching in conjunction with replacing, rarely alone. But imagine you’re trying to search for a specific string in JavaScript, but forgot which String delimiter (‘ or “) you used. Here’s the grep command:

grep -R "[\"']Some string[\"']" /path/to/your/webapp

Naturally, you don’t have to grow a beard and become a CLI geek to harness the power of regular expression searching, here’s how you do the exact same thing in Eclipse:

Regular expression search in Eclipse

Replacing

As mentioned above, I use regular expressions mostly for searching and replacing, a very powerful technique that saved me countless hours of mind-numbing, repetetive typing. Have you ever heard a co-worker make the same keyboard sound many times in a row? Like moving the cursor to the next line, placing it at the beginning and pressing CTRL+V? I’m a lazy person, and I can’t stand repetetive typing tasks. Fortunately, you can avoid the majority of these with regular expressions.

Here’s an example of how regular expression search and replace speeds up refactoring. We had a whole lot of test cases that looked like this:

assertThat(RomanNumerals.convert(1), is("I");
...
assertThat(RomanNumerals.convert(5), is("V");
...
assertThat(RomanNumerals.convert(10), is("X");

Too much duplication, so we created a method assertRomanNumeralEquals() to get rid of that:

private static void assertRomanNumeralEquals(String roman, int arab) {
    assertThat(RomanNumerals.convert(arab), is(roman));
}

Eclipse was able to extract the method for us, but it wasn’t able to make all the assertThat() invocations use the new method instead. So that’s where regular expression replacement comes in handy even in a sophisticated IDE. I replaced the following expression:

assertThat\(RomanNumerals.convert\((.*)\),\ is\((".*")\)\);

With this:

assertThatRomanNumeralEquals(\2, \1);

This is how it looks in Eclipse (select the lines to which you want to apply this before opening the find/replace dialog):

Regular expression search and replace in Eclipse

The expression might look a bit intimidating if you’re not used to regular expressions, but you will be able to write down something like this in no time if you practice them.

In case you’re wondering, this is also possible on the command-line, with the sed command.

Conclusion

Regular expressions are a powerful tool for processing and editing text, automated or interactively. If you use them habitually, you will have learned something for life, because every reasonable editor and IDE supports them. However, regular expressions are not standarised, so there are slight differences between Perl, Java etc. You might have noticed that there  are also some minor differences between grep and Eclipse in the first example above. This is sometimes good for a few short confusions, but it has never hurt my productivity notably.

Speaking of productivity; although regular expressions will probably not make you write code faster, they can significantly increase refactoring speed, a task on which I find myself working most of the time. How much time do you spend actually writing code down? And how much time do you spend editing existing code? I think the ratio is at least 1:10 in my case. If you are able to refactor fast, you will refactor more often, which is likely to improve design and maintainability of your code.

If you, however, decide to ignore regular expressions until you find a situation in which you really need them (that might never happen, you can always find a workaround), you are entering a negative feedback loop: You are not very familiar with them, so if you are faced with problems, they don’t come to mind and you don’t use them. If you don’t use them regularly, you will never become familiar with them. Searching and replacing is an ideal way to break that loop, so I suggest you try it.

Fighting RSI (Part 3)

It’s been a while since I last experienced RSI related pain as described in my Fighting RSI series of blog posts, and that’s probably why I focused on other things and never got round to writing part 3. But I’ve been asked to do so several times in the last few months, and I might as well wrap the whole issue up right now, so here it is. It’s not bad that I waited so long though, because by now I’ve been using my new input devices intensively for several months.

Kensington Orbit with Scroll Ring

In part 1, I investigated ergonomic mice, trackballs in particular, and ended up buying the Kensington Orbit with Scroll Ring.

It’s a good trackball, but I’m not so sure if it’s really better for my hand than a regular mouse. When using it for several hours in a row (mostly while playing games), my hand does get a little tired. But other than that, I’m very happy and I don’t have much to add to part 1. I got pretty good at using middle mouse button emulation (i.e. pressing both the left and the right button at the same time), works all the time now.

Kinesis Freestyle

Since I wasn’t convinced that the mouse was the sole cause of my pain, I looked into ergonomic keyboards in part 2, and decided that the Kinesis Freestyle was the best in my price range. I ordered one directly with Kinesis and soon got my hands on it.

A very nice keyboard. The material feels durable and expensive – the difference to cheap keyboards is vast. The keys have just the right resistance and make just the right noise (neither silent nor annoying) and it looks pretty slick. So far, so, good – I didn’t expect a cheap piece of hardware for this kind of money.

One of the reasons I decided to buy the Kinesis was that it is a modular keyboard (I love modular stuff), which means that you’ll buy a base keyboard to which you can add accessories that will let you position the keyboard in an ergonomic way.

I tried to be cheap and ordered just the base keyboard at first, which turned out to be a bad idea. The Freestyle without any accessories is not an ergonomic keyboard, although it’s still a pretty cool keyboard if that’s not what you’re looking for. I tried to figure out which accessory to buy by arranging the keyboard with books, but it didn’t really work. I eventually went with the most flexible and popular option, the VIP:

It features wrist rests which are absolutely essential (trust me on this) and makes it possible to adjust the keyboard’s angle. There are only two settings, but that’s enough for me:

With the VIP accessory, the Freestyle is a wonderful ergonomic keyboard. You can easily adjust it to a comfortable typing position at any time. If a non-touch typist insists on using your computer, you can just move the parts together:

You can also move the parts far apart, I can’t think of any other keyboard that lets you do that. I’ve even seen one guy who ordered a Freestyle with an extra long chord and mounted each part on each side of his ergonomic chair – quite impressive. You can really go nuts with this, e.g. place your mouse between the parts:

I usually have them close to each other, but when I’m typing with my baby son in my arms, I move them a bit further apart.

I decided to buy the US version of the keyboard because the German version was only available from German resellers which sold it at ridiculously high prices. That worked pretty well for me: Programming with the US layout makes much more sense (and fun), and thanks to the US international layout, I can still type special German letters effectively. It was a bit difficult at first to use a German layout at my laptop and at work and an US layout at my desktop computer, but I can mentally switch layouts seamlessly by now.

The Freestyle doesn’t have a num pad, which was actually one of the reasons why I decided to buy it. The num pad consumes valuable space on the desk, forcing me to either not center the keyboard in front of me or to reach unreasonably far for the mouse. As a touch typist, I hardly used the num pad anyways, so I was glad to get rid of it. The Freestyle does have an Fn key that will make a couple of other keys function as the num pad keys, but I’ve never used that, except accidentally.

Speaking of which, there are a couple of special keys on the left:

I never use those, but I guess they just had some free space there, I don’t mind. All of these are hardwired to key combinations, so they work on pretty much every OS. I thought that, as an Emacs user, I could make good use of the copy and cut key (C-C and C-X, see?) but they are too difficult to reach from the normal typing position.

As you can probably tell by now, I’m pretty happy with the Freestyle, and it was definitely worth its money in my case, because the pain disappeared after a short while.

One last piece of advice: If you are going to buy such an expensive keyboard, make sure to invest the additional $10 to buy a cover for keeping it clean:

Disclaimer

Please don’t mistake me for an RSI expert (in the unusual event that I make that impression), I’m just a geek with pain, which is a dangerous combination. I’ll admit that I suspected that my pain was mostly caused by the mouse, a cheap piece of hardware, yet I looked into the considerably more expensive ergonomic keyboards, mostly because I think they’re cool :). It turned out well for me, because the pain was indeed caused by the keyboard. I was lucky with my choice and the pain did go away. However, I guess the best thing to do with RSI pain is to go see a physician and figure out what’s causing the pain, then solve that problem.