OSCON 2007 Trip Report

by anandeep on August 17, 2007 01:35pm

My overall impression was that OSCON was lower key than last year. There seemed to be fewer booths in the Exhibition floor and less palpable excitement in the venue. A lot of people were complaining about the quality of the tutorials and the talks. Or it may just be that this was my second time around attending OSCON and it didn’t have the same quality of excitement for me compared to the very first time!

Microsoft’s name was on the bag that they gave us. That was good to see. Also Microsoft’s name was on the marquee as a diamond sponsor. The other sponsors were all big names. Intel was a diamond sponsor as was Zimki. Zimki? Wait a minute, Zimki isn’t a big name! How did an open source web development platform company spend so much money? According to their web site “Zimki is Fotango's flagship product, a JavaScript web application development platform that enables developers and businesses to build and deploy web apps quickly without the need for tools, equipment or hosting. With no set-up fees and subscription charges,users simply pay for the computing resources that they consume on a utility basis.” Their blog also states that they were holding back their plans to roll out Zimki at OSCON . So a small company spent all this money and had no presence at OSCON. Shades of the dot com boom. Hopefully open source web development platforms are not today’s dot coms!

I attended two keynotes (that I remembered an hour after I attended them) – one involved Eben Moglen, the lawyer dude for the FSF, tearing into Tim O’Reilly. Tim O’Reilly was asking Eben questions about whether GPL V3 gave Google a free ride. Eben went into how he wanted to protect freedom and how the Open Source people had “wasted ten years” not making “freedom” the issue. But as persuasive and articulate as Eben is, I think he left the feeling with the audience that the FSF had given in to Google to get GPL V3 passed. Eben even used the words “diplomacy” to describe the process of building GPL V3. 

Some dramatic moments such as Eben pointing to Tim and then to the sign behind him and saying,” Take down that sign with YOUR name on it and put “Freedom” there instead”. Tim even went to say to him – “I will ignore the personal attacks”. To which Eben said – “This is not a personal attack, it’s an invitation to diplomacy”. Wow! I could have watched a musical and not had so much drama.

Don’t get me wrong, I admire the FSF’s devotion to its cause. They have been consistently practicing what they preach. I am glad that there are people like them to keep even big companies like Microsoft honest. But I feel let down with their inconsistency with respect to Google.

The other keynote that I enjoyed was Bill Hilf. Bill is our GM, and we know all the stuff he said in speech. We try not to say one thing and practice another (surprise, surprise!). But he said it all so clearly, and so well in front of a largely skeptical audience. It was a masterful and engaging performance. Even when ambushed by Nat Torkington with questions that were not on the agenda, he didn’t lose his verve and kept on emphasizing what we do instead of what we say. It feels great to have a official place in the Microsoft firmament @ www.microsoft.com/opensource . Wait! Or was it nice to be the exclusive open source guys in such a big company? (From a purely selfish point of view)

I attended a few talks. One was Sam Ramji’s talk about our Interop efforts in virtualization, identity and management. To tell you the truth, I went because Sam is my boss. But I stayed because he did a great job of simplifying and presenting the information. I learnt and reinforced a ton about virtualization and the Interop challenges around it. I now have a firm grasp on the subject – not isolated chunks of information unconnected to each other. Ok Sam, when are you doing a talk on High Performance Computing? J

The other significant talk I attended was about Hadoop. Hadoop is an open source software platform that lets one easily write and run applications that process vast amounts of data. Or basically it implements Google’s MapReduce. According to Google’s original paper on MapReduce - “Programs written in the MapReduce style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system”.

Hadoop has been created by Doug Cutting, the creator of Lucene and Nutch. In order for him to do MapReduce effectively he had to do a “Google File System (GFS)” like system called “Hadoop Distributed File System (HDFS)”. HDFS was originally built as infrastructure for the Apache Nutch web search engine project.

Now, Yahoo is using Hadoop and HDFS for its back end. There is now an open source implementation of Google’s Open Source based proprietary stuff. If the community get’s behind it, it may be that the truly open source stuff may outshine the open source but proprietary stuff. Makes your head spin.

Oh, and why the name Hadoop? Doug Cutting’s son’s favorite elephant was named Hadoop. A name that came from the son’s imagination. I love Open Source!