• 

  

• Now available

  • 

• Follow me

  Follow me on Twitter

• Download DB2

  • 
• Search for:

• Recent Posts

  • Reasons to switch to DB2 9.7
  • Apple “Genius”
  • DB2 Express-C 9.7 and the Django adapter released
  • Selling my old MacBook Pro
  • Discussing code reading and testing with Dave Thomas, Cory Foy and Tim Bray
  • Do Androids Count Electric Sheep with DB2 or MySQL?
  • Of labels and limits
  • Announcing ThinkCode.TV’s new English newsletter
  • TextMate bundle for DB2
  • Memoization in Ruby and Python

• Recent Comments

  • Surachart Opun on Reasons to switch to DB2 9.7
  • Robert Bath on Apple “Genius”
  • Roderick van Domburg on Reasons to switch to DB2 9.7
  • Matt Zago on Apple “Genius”
  • sambeau on Apple “Genius”

• Categories

  • .NET (13)
  • Books (29)
  • Cocoa and Objective-C (6)
  • DB2 (47)
  • Django (22)
  • Erlang (4)
  • General (60)
  • Hardware (3)
  • Haskell (6)
  • Mac (26)
  • Mathematics (3)
  • Merb (10)
  • Photography (1)
  • Python (28)
  • Quick Tips (16)
  • Reviews (3)
  • Ruby (87)
  • Ruby Benchmark Suite (10)
  • Ruby on Rails (84)
  • Screencasts (3)
  • Startup (8)
  • This Week in Ruby (15)
  • Zenbits (3)

• Blogroll

  • Anant Jhingran
  • ChannelDB2
  • DB2 Express-C
  • DB2 on Rails Blog
  • Django
  • FreeDB2
  • Giovanni Intini
  • Haskell
  • Irshad Raihan
  • My Math Blog
  • Passiomatic
  • Planet DB2
  • Programming Screencasts
  • Python
  • RickLyn
  • Ruby
  • Ruby on Rails
  • Stacktrace.it (ITA)

The most effective martial artists specialize in their discipline, but are not afraid to cross-train in others. Bruce Lee—arguably the most famous and influential martial artist of the past century—trained first in Tai Chi Chuan, then Gung Fu, and boxing, as well as learning western fencing. The insight taken from so many disciplines led him to create the Jeet Kune Do form of combat.

Programmers are not all that different. Cross-training in other languages and frameworks can only improve one’s overall mastery of the craft. When it comes to Ruby frameworks, the two most popular choices are Ruby on Rails and Merb. They’re often seen as being contenders, but this truly isn’t a zero-sum game; learning both is a very sensible move. They both enable you to write web applications in Ruby, and are somewhat similar, so learning one after you know the other shouldn’t be very challenging. In the many cases people learn Merb after they’ve had some experience with Rails, but either way, acquiring a solid grasp of both frameworks provides developers with extra flexibility. Often people who learn both, will end up mostly just using one or another, depending on their individual preferences. But it’s worth knowing them so as to be able to write both CRUD-style applications that fall within Rails’ solution space, and more complex, edge cases where Rails’ opinions will end up contending with yours.

Among the reasons to give Merb a chance, is its focus on performance, a smaller memory footprint and an extreme level of modularity, which enables you to pick and choose which components you’d like to use.

Merb is not as mature as Rails, of course, but it has reached version 1.0.x and with it developers can have greater confidence in a more stabilized API. Now is perhaps the best moment to get involved and learn more about this rising framework. Not surprisingly though, Merb finds itself in a similar spot to the one that Rails was in a couple of years ago (in terms of weakness of documentation when it comes to getting started). Thankfully, this point is being taken seriously and there’s been some major progress in terms of improving the documentation for Merb. Below are some useful links to get you started with Merb.

Merb has an official API documentation, a wiki, a google group, and a community site called Merbunity for news, projects and tutorials. The irc.freenode.net #merb channel is also a useful and welcoming spot. Furthermore, there is a Peepcode PDF draft called Meet Merb. If you want something even more substantial, on the book front there are several titles coming out in the near future. These include Merb in Action, The Merb Way, Beginning Merb and Merb: What You Need To Know. There is also an open source Merb book, whose development is led by Matt Aimonetti. It’s a work in progress, but probably a very good starting point, which just happens to have the added bonus of being free. And if your interested in Merb, don’t miss InfoQ’s interview with Yehuda Katz, who’s Merb’s lead developer and one of the sharpest guys we have in the Ruby community.

Finally, if you are a professional developer who wants to quickly progress with Merb and bring their skills to the next level, do not miss your chance to attend a three day intensive course on Merb, which is being offered by Yehuda and Matt in Phoenix, AZ between January 19 and 21 (2009). Registration has been open for two days already and 20 out of the 30 available spots have already been snapped up. The remaining seats won’t last more than a day or two, so if you are interested, don’t delay (sign up now and you’ll also benefit from an early registration price).

2009 is almost here, so why not take the opportunity to learn Merb this year?

The long awaited Ruby virtual machine shootout is here. In this report I’ve compared the performances of several Ruby implementations against a set of synthetic benchmarks. The implementations that I tested were Ruby 1.8 (aka MRI), Ruby 1.9 (aka Yarv), Ruby Enterprise Edition (aka REE), JRuby 1.1.6RC1, Rubinius, MagLev, MacRuby 0.3 and IronRuby.

Disclaimer

Just as with the previous shootout, before proceeding to the results, I urge you to consider the following important points:

• Engine Yard sponsors this website, and also happens to sponsor, to a much greater extent, the Rubinius project. Needless to say, there is no bias in the reporting of the data below concerning Rubinius;
• Don’t read too much into this and don’t draw any final conclusions. Each of these exciting projects has its own reason for being, as well as different pros and cons, which are not considered in this post. They each have a different level of maturity and completeness. Furthermore, not all of them have received the same level of optimization yet. Take this post for what it is: an interesting and fun comparison of Ruby implementations;
• The results here may change entirely in a matter of months. There will be other future shootouts on this blog. If you wish, grab the feed and follow along;
• The scope of the benchmarks is limited because they can’t test every single feature of each implementation nor include every possible program. They’re just a sensible set of micro-benchmarks which give us a general idea of where we are in terms of speed. They aren’t meant to be absolutely accurate when it comes to predicting real world performance;
• Many people are interested in the kind of improvements that the tested VMs can bring to a Ruby on Rails deployment stack. Please do not assume that if VM A is three times faster than VM B, that Rails will serve three times the amount of requests per minute. It won’t. That said, a faster VM is good news and can definitely affect Rails applications positively in production;
• These tests were run on the machines at my disposal, your mileage may vary. Please do test the VMs that interest you on your hardware and against programs you actually need/use;
• In this article, I sometimes blur the distinction between “virtual machine” and “interpreter” by simply calling them “virtual machines” for the sake of simplicity;
• Some of the benchmarks are more interesting for VM implementers than for end users. That said, if you think the benchmarks being tested are silly/inadequate/lame, feel free to contribute code to the Ruby Benchmark Suite and if accepted, they’ll make it into the next shootout;
• Finally, keep in mind that there are three kinds of lies: lies, damned lies, and statistics.

Ruby implementations being tested

All of the Ruby implementations that were able to run the current Ruby Benchmark Suite have been grouped together in one main shootout. This group consists of Ruby 1.8.7 (p72, built from source, and installed through apt-get), Ruby 1.9.1 (from trunk, p5000 revision 20560), Ruby Enterprise Edition (1.8.6-20081205), JRuby 1.1.6RC1 and Rubinius (from trunk), all of them were tested on Ubuntu 8.10 x64, plus Ruby 1.8.6 (p287. from the One-Click Installer) on Windows Vista Ultimate x64. The hardware used for this benchmark was my desktop workstation with an Intel Core 2 Quad Q6600 (2.4 GHz) CPU and 8 GB of RAM. JRuby was run with the -J-server option enabled and by specifying 4 Mb of stack (required to pass certain recursive benchmarks). The best times out of five iterations were reported, and these do not include startup times or the time required to parse and compile classes and method for the first time. Several of these new tests also have variable input sizes.

The MagLev team provided me with an early alpha version of MagLev for the purpose of testing it in this shootout. Since this VM is not mature enough yet to run the Ruby Benchmark Suite, I used custom scripts against an old version of the Ruby Benchmark Suite on Ubuntu 8.10 x64. MagLev was tested, along with Ruby 1.8.6 (p287), on the same machine as that of the main shotoout, though the benchmarks were different (even when they had the same names as the ones in the main shootout).

MacRuby 0.3 and Ruby 1.8.6 (p114) were tested on Mac OS X Leopard using the previous version of the Ruby Benchamrk Suite. Since my MacBook Pro died (sigh), for this benchmark I used a Mac Pro, with two Quad-Core Intel Xeon 2.8 Ghz processors and 18 GB of RAM.

IronRuby (from trunk) and Ruby 1.8.6 (p287) were tested on a previous version of the Ruby Benchmark Suite on Windows Vista x64 on the same quad-core used for the main shootout. The MagLev, MacRuby and IronRuby numbers reported here were the best times out of five iterations, and include startup time. IronRuby on Mono was not tested because I couldn’t get it to work on my machine, despite having tried several IronRuby versions and two different Mono versions. Please also notice that Ruby 1.8.6 (p287) was tested twice on Windows, once for the main shootout against the current Ruby Benchmark Suite, and a second time to compare it with IronRuby, against the old benchmarks.

Note: As tempting as it is, do not compare implementations that belong to different shootouts directly to one another. It would be very disingenuous to directly compare VMs tested with different benchmarks and/or different machines. The only comparisons that make sense are the ones within each of the four groups.

Main shootout

The following table shows the run times for the main implementations. The table is fairly wide, so you’ll have to click on the image to view the data in a new tab.

Green, bold values indicate that the given virtual machine was faster than Ruby 1.8.7 on GNU/Linux (our baseline), whereas a yellow background indicates the absolute fastest implementation for a given benchmark. Values in red are slower than the baseline. Timeout indicates that the script didn’t terminate in a reasonable amount of time and was (automatically) interrupted. The values reported at the bottom are the total amounts of time (in seconds) that it would take to run the common subset of benchmarks which were successfully executed by every virtual machine. When our baseline VM generated an error, others were used, starting with Ruby 1.8.7 on Vista (for color coding purposes only).

The following image shows a bar chart of the total time requested for the common subset of successfully executed benchmarks (those whose names are in blue within the tables):

More interestingly, the following table shows the ratios of each Ruby implementation based on the baseline (MRI):

The baseline time is divided by the time at hand to obtain a number that tells us “how many times faster” an implementation is for a given benchmark. 2.0 means twice as fast, while 0.5 means half the speed (so twice as slow). The geometric mean at the bottom of the table tells us how much faster or slower a virtual machine was when compared to the main Ruby interpreter, on “average”. Just as with the totals above, only those 101 tests, which were successfully run by each VM, where included in the calculation.

More concisely, here is a bar chart showing the geometric mean of the ratios for the various implementations tested:

I prefer to let the data speak for itself, but I’d like to briefly comment on these results. Just a few quick considerations.

Working off of the geometric mean of the ratios for the successful tests, Ruby MRI compiled from source is twice as fast than the Ruby shipped by Ubuntu, and by the One-Click Installer on Vista. The huge performance gap between ./configure && make && sudo make install and sudo apt-get install ruby-full should not be taken lightly when deploying in production. These numbers also reveal what most of us already knew: Ruby is particularly slow on Windows (800-pound gorillas in the room, or not).

Performance-wise Rubinius has more work left to be done to catch up with Ruby 1.8.7 and other faster VMs, particularly if we take into account the number of timeouts. But it has improved in the past year and I think it’s on the right track.

Ruby Enterprise Edition is about as fast as Ruby 1.8.7 compiled from source, which is reasonable considering that it’s a patched version of Ruby 1.8.6 aimed at the reduction of memory consumption (a parameter which wasn’t tested within the current shootout).

Speaking of excellent results, Ruby 1.9.1 and JRuby 1.1.6 both did very well. It looks like we finally have a couple of relatively fast alternatives to what is a slow main interpreter. According to the results above, and with the exception of a few tests, on average they are respectively 2.5 and 2 times faster than Ruby 1.8.7 (from source), and 5 and 4 times faster than Ruby 1.8.7 installed through apt-get on Ubuntu or Ruby 1.8.6 installed through the One-Click installer on Vista. Again, this does not mean than every program (particularly Rails) will gain that kind of speed, but these results are very encouraging nevertheless.

MagLev

There has been a lot of buzz about MagLev since Avi Bryant’s first benchmarks were shown a few months ago. Here we finally see it being put to the test. The table below shows the times obtained by running MagLev and Ruby 1.8.6 (p287) against MagLev’s set of benchmarks based on the old Ruby Benchmark Suite:

And here are the ratios:

You’ll notice how MagLev swings from being much faster than MRI to being much slower. I believe there is much room for improvement, but at almost twice the speed of MRI, these early results are definitely promising.

MacRuby

These are the times for MacRuby 0.3 on Mac OS X 10.5.5:

And of course, the ratios against the MRI baseline:

MacRuby is relatively new, so these are not bad results. More work is required, but it’s a good start.

IronRuby

Finally (I promise these are the last ones), here are the two tables for IronRuby and Ruby 1.8.6:

IronRuby is slower than Ruby 1.8.6 on Windows, which in turn is much slower than Ruby 1.8.7 on GNU/Linux. This is not very surprising. This project has been focusing on integrating with .NET and catching up with the implementation of the language by improving the RSpec pass rate, as opposed to performing any optimizations and/or fine tuning (as per John Lam’s presentation at RubyConf 2008). We’ll measure its improvements in the next shootouts.

Conclusion

Overall I think these are great results. Ruby 1.8 (MRI), with its slowness and memory leaks, belongs to the past. It’s time for the community to move forward and on to something better and faster - and we don’t lack interesting alternatives to do so at this stage.

I hope that for the next shootout, MagLev, MacRuby and IronRuby will be able to run the benchmark suite, so that they can all be tested and directly compared with each other. I also hope to include Tim Bray’s XML benchmark, some sort of “Pet Shop” sample Rails and Merb application and, above all, include memory usage statistics.

You can find the Excel file for the main shootout here. That’s all for now. Feel free to comment, subscribe to my feed, share this link and promote it on Hacker News, Reddit, DZone, StumbleUpon, Twitter, and Co. Putting together this shootout was a lot of work, so I definitely appreciate you spreading the word about it. Until next time…

Update (December 10, 2008): This article has been updated to correct a couple of major issues with yesterday’s results. I adjusted my commentary as well, in light of the corrected figures.

Update (February 7, 2009): Thanks to Makoto Kuwata, a Japanese version of this article was published in the Rubyist Magazine.

Zenbits are posts which include a variety of interesting subjects that I’d like to talk about briefly, without writing a post for each of them.

Merb: A few days ago Merb 1.0 was released. Congratulations to Ezra Zygmuntowicz on this important milestone, the Merb community and Engine Yard (who finances the project). Merb 1.0 wasn’t even out yet when some people had already started commenting on the fracturing of the Ruby community that this new framework might bring with this, and the impact that this high visibility “competitor” might have on Rails. I believe that having more than one widely adopted web framework will only benefit the Ruby community. Furthermore, it’s important to remember that this is not a zero-sum game. Ruby programmers are perfectly capable of learning two frameworks and using one or the other, depending on the project at hand. This is particularly true if we consider that Merb, for all of its advantages - and disadvantages - when compared to Rails, is not totally different from its forerunner. If you are an expert Rails programmer, you should be able to become proficient in Merb in very little time. To help with this process, the Merb community needs to concentrate on the documentation now, given that the API is finally stable.

Rails Myths: David Heinemeier Hansson began a series of posts about Rails Myths. I like the idea of seeing common myths addressed straight from the horse’s mouth. Over the past two years, Rails has received quite a bit of backslash and old fashion FUD, so it’s important to set the record straight, whether the myths are entirely fabricated or if there is some element of truth to them. Whether you agree with David or not, it’s also nice to hear two sides of the same story. In fact, at the beginning of my book I debunk a few myths, just to set the record straight regarding what some readers may have heard surrounding the framework. It was a fun part to write.

My Book: Speaking of my book, Ruby on Rails for Microsoft Developers, I’m getting closer to the finish line. I’m about to complete Chapter 9 (out of eleven chapters). The initial schedule I was provided with has been extended slightly so that there will be sufficient time to properly review the content and ensure that it’s up to date with the final release of Rails 2.2. Some people wondered what the “Microsoft Developers” part means. Is it for people that work at Microsoft? Is it for .NET programmers? Is it for people who develop on Windows?

The truth is that “Microsoft Developers” is probably just a marketing term that Wrox selected as a catch-all for of the aforementioned categories of programmers. As an author I’m trying to serve all of them well, by providing a guide that sneaks in much of the Rails culture and softens the migration path by using an Operating System, and to a certain extent, tools that they’re already familiar with. In my opinion one of the major obstacles when switching to, or trying, Rails when coming from the Microsoft world, is the culture shock. The documentation and most books assume that you are familiar with *nix systems and tools, and this can be frustrating for those who are forced not only to learn a new language and framework, but also an entirely new set of tools. As it’s targeted at Microsoft developers, the book obviously makes quite a few references and comparisons to the .NET world, where they fit. This is done so that the many .NET programmers amongst the group of so called “Microsoft Developers” will find the book particularly useful. Yet the book remains generic enough so that it can be used by any programmer (particularly Windows users), even those without any knowledge of the Microsoft .NET Framework or ASP.NET.

Python books: While on the subject of books, I wanted to mention that the final version of the Pylons book is available online. Despite the much less fancy UI, the book pretty much does what the Django Book did in the past. And both are available in print as well (The Definitive Guide to Django: Web Development Done Right and The Definitive Guide to Pylons). Pylons is a Python web framework that can be viewed as a Ruby on Rails clone, in a far greater way than Django could ever be considered.

Another thing I want to mention is that I received a copy of Expert Python Programming. I haven’t gotten to far into it yet, but from what I’ve seen so far, things look good. I hope to be able to read it through, over a weekend in the near future and then provide a proper review. Stay tuned.

Language Popularity: If you take a look at the TIOBE Index, you’ll notice a few interesting things: Ruby has dropped two positions since last year, and it’s now the 11th most popular language in the world. This shouldn’t be cause for concern though, as shown by this Ruby graph. Python on the other hand is increasing in popularity and moved from the 7th to the 6th most popular language. Interestingly, according to the index (the results of which are educated guesses only), Python would seem to be more popular than C#. I find this to be true, in terms of online activity within an increasingly vibrant community, but in my opinion, the job market hasn’t caught up yet. In fact, at least in Toronto, when there’s a Python opening it’s pretty much an event that’s worthy of being discussed on the local Python mailing list. C# openings are much more common. This may be different in Silicon Valley, of course. It would also seem that Delphi has experienced a huge come back, moving from the 11th position last year to the 8th one this time around. It’s hard to imagine that Delphi has had a similar level of adoption as C# and thus has become more popular than Perl, JavaScript and Ruby. Delphi is a great solution for Win32 programming, but I don’t quite believe this overly optimistic outlook. And if this is the case, where are all the Delphi jobs and buzz?

DB2: This interview shows a few good reasons why even smaller and medium sized companies are increasingly adopting DB2. And while the video doesn’t mention it, IBM is coming out with an updated version of DB2 Express-C 9.5. This new version, 9.5.2 or 9.5 FixPack 2, is going to introduce exciting new features, including an engine for full text search.

The Great Ruby Shootout These days you hear a lot of talk about parallel programming. Intel promotes it and despite their bias, it’s plausible that parallel programming will become important as the CPU market heads towards an increasingly larger number of cores, as opposed to focusing on the frequency of said CPUs. In the world of Ruby, this translates into multiprocessing, as opposed to multithreading due to the infamous GIL (Global Interpreter Lock). This means that Ruby will most likely approach the problem similarly to how Python 2.6 did with the multiprocessing module, which is a process-based interface. The obvious exceptions are JRuby and IronRuby, which establish a 1 to 1 relationship between green threads and OS threads.

For the shootout, it would be interesting to see some multithreaded code, so as to get a better sense of how well JRuby and IronRuby compare to MRI and 1.9, when more cores are available. In fact, the long-promised shootout will be performed on a quad-core machine with 8GB of RAM. If Charles Nutter, John Lam, or any of their team members would like to contribute some programs that are able to take advantage of “native” multithreading, I’d be very happy to include them in the Ruby Benchmark Suite, to be used for my shootout.

The repository requires some love and refactoring, since it needs to be split in two types of benchmarks. The simpler one will evaluate the execution time minus the startup time, while the more advanced benchmark will also exclude the time required for parsing and loading modules, classes and methods in the AST. It would also be nice to test each program with variable input sizes and report these results accordingly. Right now I’m very busy with the book, but as I become more available, I’ll start working on this.

Finally, I want to point out a very interesting article about performance and UIs. Slow is indeed a very relative concept, and it’s important to understand how to analyze and respond to the user requirements when it comes to the responsiveness of an application as a user interacts with it.

Hardware: I finally bought a Trackball made by Logitech and the Microsoft Ergonomic Keyboard (Microsoft makes great hardware). I don’t have wrist problems, but I’d like to see how these two affect my extensive computer usage. I plan to report my experience as soon as I’ve had a chance to use these input devices for a while, since I know this is a topic that interests lots programmers (many of whom end up being victims of RSI, and some of the IRS :-P). I also bought a bad-ass color laser printer which is quite handy when you’re a programmer and you are writing a book. I’ll let you know how it goes. What I didn’t buy, but still think is awesome, is the Flip minoHD. It’s the equivalent of an iPod for the world of camcorders. $235 for a camcorder that’s so perfectly compact, and yet that can record in HD, is a pretty sweet deal. I’m considering it for Christmas, assuming it reaches Canada by then.

Next Page →