Jun

19

DB2 Express-C 9.7 and the Django adapter released

Filed Under DB2, Django, Ruby, Ruby on Rails | 9 Comments

This is a great day for those of us who love DB2, as DB2 Express-C 9.7 has just been released. As mentioned before, this is the best DB2 ever, and an extremely important release.

To learn more about what’s new in this release, please check out the recording of our latest webinar:


If you run Linux, Unix or Windows, download it while it’s hot.

DB2 9.7 on the Cloud

Another great aspect of this release is that for the first time ever, DB2 has been released both as a product and as a deployment on the Cloud. If you pop over to RightScale, you can get a trial account for free and should see DB2 Express-C 9.7 on both CentOS and Ubuntu within the partner catalog. RightScale has been an amazing partner and they really do wonders to simplify Cloud Computing. In ten minutes time you can be up and running on the Cloud, thanks to the templates provided.

DB2 on the Cloud

DB2 support for Django

But the good times don’t stop there, we are also announcing the first official release of the Django adapter for DB2. It sounded crazy when I first proposed the idea within IBM back in 2006, but now it’s a reality.

You can download the .tar.gz archive from the Google Code homepage for the project, or simply by clicking here. This version fully supports the Django 1.0.2 API. For instructions on how to install it, please read the Getting started with the IBM DB Django adapter guide. The current version supports DB2 for Linux, Unix, Windows and MAC OS X, version 8.2 or higher (9.5 FP2 or higher for MAC OS X). In the future, IBM Cloudscape, Apache Derby, Informix (IDS) and both System i & z/OS will be supported.

ibm_db gem updated to 1.1

I’ll conclude this DB2-centric post with a smaller, but still interesting announcement. The ibm_db gem has been updated to version 1.1. This release includes support for ActiveRecord’s QueryCache mechanism, enhanced support for BigInt (and BigSerial), support for rename_column (requires DB2 9.7), parametrization of the timestamp datatype (requires DB2 9.7), and a few fixes and performance enhancements as well. It is recommended that you upgrade to this version.

Jun

5

Do Androids Count Electric Sheep with DB2 or MySQL?

Filed Under DB2, Mac, Ruby, Ruby on Rails | 26 Comments

Counting rows is an ubiquitous operation on the web, so much so that it’s often overused. Regardless of misuse, there is no denying that the performance of counting operations has an impact on most applications. In this post I’ll discuss my findings about the performance of DB2 9.5 and MySQL 5.1 regarding counting records.

For those of you who are not into science fiction, let me clarify that the odd title of this post is a tongue-in-cheek reference to the great novel, Do Androids Dream of Electric Sheep?.

I connected to the database, created the table, imported the data and benchmarked counting operations using ActiveRecord in a standalone script. Here is the code I used:

#!/usr/bin/env ruby
require "rubygems"
require "active_record"
require 'benchmark'

ActiveRecord::Base.establish_connection(
  :adapter  => :mysql,
  :username => "myuser",
  :password => "mypass",
  :database => "mydb")

ActiveRecord::Schema.define do
  create_table :people, :force => true do |t|
    t.string :name, :null => false
    t.string :fbid, :null => false
    t.string :gender
    t.string :profession
  end
end

class Person < ActiveRecord::Base
end

# This can be sped up by performing an import instead
Person.transaction do
  File.open("person.tsv").each_line do |line|
    line = line.split(/\t/)
    p = Person.new
    p.name = line[0]
    p.fbid = line[1]
    p.gender = line[6]
    p.profession = line[17]
    p.save!
  end
end

n = 100
Benchmark.bm(26) do |x|
  x.report("Count all:") { n.times { Person.count } }
  x.report("Count profession:") { n.times { Person.count(:profession) } }

  x.report("Count females:") do
    n.times { Person.count(:conditions => "gender = 'Female'") }
  end

  x.report("Count males w/ profession:") do
    n.times { Person.count(:profession, :conditions => "gender = 'Male'") }
  end
end

Please note that importing records in a huge transaction containing hundreds of thousands of INSERT operations is far from the most efficient way to import. Massive imports of data using the load/import facilities provided by each database is the way to go (also see the ar-extensions plugin). The lengthy import wasn’t benchmarked here though, so it isn’t determinant for this article.

people.tsv is a 92.7 MB tab separated values file that contains 875,857 records from the Freebase project (in my file I removed the header line, leaving only records).

For those who are not familiar with ActiveRecord, the queries executed behind the scenes are (in order):

SELECT count(*) AS count_all FROM people

SELECT count(people.profession) AS count_profession FROM people

SELECT count(*) AS count_all FROM people WHERE (gender = 'Female')

SELECT count(people.profession) AS count_profession FROM people WHERE (gender = 'Male')

While the table definition (for MySQL) is:

CREATE TABLE `people` (
	`id` int(11) DEFAULT NULL auto_increment PRIMARY KEY,
	`name` varchar(255) NOT NULL,
	`fbid` varchar(255) NOT NULL,
	`gender` varchar(255),
	`profession` varchar(255)
) ENGINE=InnoDB

As easily verified by enabling logging with:

ActiveRecord::Base.logger = Logger.new(STDOUT)

Without much further ado, here are the times I obtained on my last generation MacBook Pro 2.66 GHz with 4 GB DDR3 RAM, and 320 GB @ 7200 rpm hard disk, running Mac OS X Leopard:

MySQL:

  Count all:                  42.467522
  Count profession:           52.130935
  Count females:              54.575469
  Count males w/ profession:  64.046631

DB2:

  Count all:                  5.818485
  Count profession:           7.714391
  Count females:              8.556377
  Count males w/ profession:  9.656739

Or in graph form:

COUNT performance graph

That’s an impressive difference. To be exact, in this example DB2 was between 6 and 7 times faster than MySQL. In the case of COUNT(*), DB2 counted almost a million records in 58 milliseconds, or in about the blink of an eye according to Wolfram Alpha.

For those who are skeptical, please note that DB2 was not manually fine-tuned in any way. The client codepage was set to 1252 to allow Greek letters, and the log size was increased to permit such a huge transaction during the import. That’s it, no optimizations were attempted. This is DB2 Express-C out of the box. It looks like smart androids count electric sheep with DB2 after all. :-P

The advantages of DB2 over MySQL when dealing with a massive volume of traffic are well known (and not limited to performance either), but DB2 can dramatically improve performance even for your average web application. And DB2 9.7, which will be released this month, increases the performance and the ability to self-tune itself to the available resources and required workload even further. If you’d like to try DB2 Express-C for yourself, you can download it here. It doesn’t cost you a dime to obtain and can be used for development, testing and production absolutely free of charge.

May

16

Startup Interviews: Zooppa.com

Filed Under Ruby on Rails, Startup | 2 Comments

What follows is an interview with Nicholas Wieland, CTO of Italy-based Zooppa, a fast growing social network for creative types. This is the second in a series of interviews I will carry out with interesting figures from the micro-ISV and startup scene. If you have a compelling story to tell, own or run a tech startup, and would like to be featured, please drop me a line via email.

Nicholas Wieland

1. I’d like to start by tracing your background. What did you do prior to Zooppa?

Programming, programming, programming. I started out working for Neato Europe as their “one-man-band programmer”, building their e-commerce site and taking care of their infrastructure (nothing more than a bunch of servers). In those days my love of FreeBSD, PostgreSQL and dynamic languages started to grow exponentially. As the only employee I had to find a way to produce quantity and quality at the same time, and the technologies above were a perfect fit for the job.

After leaving Neato I did consulting work for a few years, for customers like Monte dei Paschi di Siena and Italian Telecom. Albeit those are big names, it was with the smaller scale customers that I enjoyed working the most. I won’t mention them by name though as they don’t impress people as much as big banks. :) I used to work primarily with Python, while developing a growing interest in Ruby, which blossomed when Ruby on Rails was released.

In the meantime I was helping Marco Ceresa writing the first book in Italian about Ruby (thus cheating on Python officially for the first time) and working part-time for Assembla on their Hotchalk project.

2. What led to the creation of Zooppa and how did you become its CTO?

Zooppa came from an idea that Davide Lombardi, an Italian journalist, had to found a company with the help of Riccardo Donadon and his firm, H-Farm, an early stage startup incubator. I joined Zooppa almost by accident, having answered a job posting thinking that it was some kind of consulting position for RoR from a forward-thinking employer. I soon found myself talking with Riccardo and Peter Caiazzi (of Netscape fame) at a wonderful countryside farm, full of fellow geeks. I was immediately sold.

I was first hired as a consultant and a few months later become the CTO. I guess the reason is that I’m a good programmer. You should ask Riccardo. :)

3. Can you explain to our readers who may not familiar with it, what Zooppa is? (Think an elevator pitch.)

Zooppa is a web application for designers and other creative types. It’s based on a new approach to online advertising and on a value proposition that goes beyond the simple “click” formula. We give our users a brand that is looking for a commercial, they create ads through a competition (videos, print ads, radio commercials, banners or simply concepts), and these ads are voted on by the site’s community. Eventually the ad makers win money (I think we’ve awarded $400,000 to date) based on the popular vote and the choice of the customer who commissioned the project.

In my opinion it’s a wonderful way for people to show what they’re able to accomplish for companies like Google, Nike, TomTom and many others. And based on the number of people from our community who have been hired by these companies, it is clear that the companies agree as well. :)

4. What kind of funding did the company receive?

No funding other than that of H-Farm. I strongly believe that you need to have a product before you ask for funding. I’m absolutely against the “promise-based” economy we all know. We’re going to be looking for our first round of substantial funding very soon, as we’re now able to show our polished product and our large community (something we couldn’t do one year ago). We have been able to give Zooppa its own character and goals, and we proved that it worked. Now we are able to make promises that we can stand behind.

5. How are things going? At this stage have you been able to be profitable, and can you disclose a few statistics about Zooppa? How many employees, users, pageviews and so on?

No, we haven’t been profitable, but we’re getting very close. We are somewhat akin to a combination of YouTube, Flickr and Facebook (in the sense that we serve videos, images and have a timeline), and we had huge startup costs, so it was basically impossible to be profitable initially, which we knew from the get-go.

We have 40,000 users more or less, 12,000 print ads, 2,000 videos and so on. It’s quite impressive when you consider that these videos are the result of a creative process and not, say, simply the recording of your cat a la YouTube. In terms of employees, we’ve got about 20 people.

6. It seems to me that the idea is innovative, the videos are fun, and there is a solid business model behind it. Why do you think the company has received little attention outside of Europe, especially by specialized blogs that cover all kinds of startups? Can you tell us about your struggle to get exposure?

Oh well, you’re opening a can of worms with that question… We made a huge mistake at the beginning: we focused on our own market without “attacking” the US one. Other than that, we didn’t receive any coverage outside of Italy, even if I did all I could to reach out to the biggest blogs that cover startups. It seems to me that you only get coverage if you are burning VC money or in some way are doing something that’s fashionable at the moment (such as with Twitter right now).

I’ve even seen one of our clones getting more coverage than us, which is kind of ridiculous as that competitor implemented the idea horribly in my opinion. I would say that the originality of the idea doesn’t really matter to such bloggers, just the funding and related issues (not to mention the various possible conflicts of interest…). Honestly this is very hard to swallow.

7. How did choosing Ruby on Rails impact the project?

We are incredibly productive. We normally code a prototype and write specs before putting it online. This is awesome, I can turn ideas into code in no time, test and refactor them later if I realize that performances are not as I would like them to be.

I strongly believe we wouldn’t survive without Rails. I still remember the presentation I gave to RailsToItaly, I just had done a complete redesign and rewrite of the site in 38 days. My presentation’s title was “Productivity with RubyOnRails or how I stopped worrying and learned to love Zooppa”. I think this title speaks for itself.

On a side note, I would also like to say that as an old school web developer and early “Railer” (you can find my posts whining about DB serials stretching as far back as 2004 with answers from DHH himself :) ). I think we have the best provider ever. I work with EngineYard and their service is terrific. I have no idea how they can offer the sort of support they do and still keep prices so reasonable. Never in 10 years on the web have I experienced the type of service I get from EngineYard. They are one of our most precious assets.

8. As CTO for the company, what are your main duties?

I code like crazy, as I already mentioned, and I take care of our architecture and its evolution. I’m in charge of other developers (http://twitter.com/oscardelben/, http://twitter.com/michelegera and http://twitter.com/smash) and provide them with specs to implement. And I whine about everything. Whining is the defining skill of every CTO, trust me. ;-)

9. Where is Zooppa heading and what plans do you have to bring it to the next level?

I want to integrate Zooppa with the major video and image platforms. And with Twitter of course. I would also like to experiment with Sinatra for a Facebook application. As well I’m coding my own blog/static site solution with Sinatra and Metal (I really dislike Wordpress). And I do so in my free time, which means BSD license all the way. I also plan to use CouchDB for our timeline.

We have a lot of stuff going, and we’re going to hire more employees, so feel free to send your CV to nwieland@zooppa.com, and be sure to include Erlang and Haskell if you are brave (I received complaints about my choice of technology, because it’s hard to find Ruby developers around here. If I start requiring Erlang and Haskell skills I will definitely establish a record number of complaints here in H-Farm). It’s going to be a lot of fun, really. :)

I really want to thank Nicholas for his time, and I invite my readers to stay tuned for more interviews. An Italian translation of this interview is available on Stacktrace.it.

Next Page →