The Decline
Overly the last few years it has become a fairly common meme in the programming community that Apache was "the place old projects go to die". The consensus being that whilst the Apache software foundation had expanded to include a massive group of projects very few of them were as overwhelmingly successful as the http server its named after or the tomcat Servlet container. Many of its projects were in an ‘incubator’ status or had a dropoff in the number of committers.
Eclipse, the other large software foundation in the Java ecosystem, was in a similar situation. Its IDE was incredibly well known and used but the the large group of spinoff projects that surrounded it had less market penetration or were focussed on more niche areas.
Furthermore the raison-d’etre of these projects was increasingly questionable in the github era. The infrastructure and hosting provided by Apache and Eclipse was unnecessary when github were willing to provide this for free to any member of the public. The motivation of the software foundation as a meeting place and community was also under question, since the social coding model that github promoted by being “a social network for code” also provided an alternative to the software foundation model.
Then Apache left the JCP's Executive Committee in quite controversial circumstances. Most people, myself included, were frustrated at Oracle’s behaviour around this issue. They were in the wrong and Apache were in the right. Of course the lack of such a prominent position also fits into the “decline” narrative that surrounded Apache.
The reports of my death have been greatly exaggerated
One would expect that given the alleged decline the Apache and Eclipse projects would slide into irrelevance. This hasn't happened and in fact both have begun to develop a wealthy ecosystem of new projects that bring in fresh blood and enthusiasm.
Apache has become a location for many of the infrastructure projects core to the big data trend such as Cassandra and Hadoop. Hadoop is a huge ecosystem in and of itself, having modularised its core its begun to evolve a series of surrounding projects such as the HBase database and the Mahout machine learning library.
Eclipse has started to bring in projects which are increasingly unrelated to its popular IDE, but certainly follow through on the goal of improving the lives of day-to-day developers. The lightweight, embeddable and popular Jetty container is now an Eclipse project as is the polyglot and asynchronous application framework Vertx.
Vertx is a particularly interesting project since its source code is still hosted on github and still uses much of the github infrastructure in order to run the project. Not only that but its a project that's close to my heart, as an occasional contributor, and day-to-day user of the Vertx project at work. When I signed off on an Eclipse contributor agreement earlier this year it was to contribute a patch to Vertx and not the Eclipse IDE, nonetheless it motivated me to help out on an Eclipse project.
The popularity of the Eclipse IDE continues to rise. I wasn't aware of this until recently but download numbers for Eclipse versions are on a pretty strong upward trend. I haven't seen numbers for the 2012 release, but here is 2009-2011:
- Indigo (2011) – 6,000,000 downloads
- Helios (2010) – 4,100,000 downloads
- Galileo (2009) – 2,200,000 downloads
Why?
The obvious question arises of why the software foundation ecosystem has changed. Why is Apache's reputation as “the place old software projects go to die" now completely undeserved and how has Eclipse been so successful at growing outside of the core domain of its IDE? I can't speak completely to the motivation of the project owners themselves but I'm certainly of the opinion that there's one thing that a software foundation brings that github can't replicate: a home for intellectual property.
A lot of people view the open source software world as something which is a hobbyist/personal project. That's true for many of us, me included. I don't work on open source projects in my day job, but I do spare time coding which is nearly exclusively open source. Of course over the last decade there's been a big trend towards open source projects being commercialised through either a support contract model or using an open source project in order to bootstrap a wider ecosystem. Red hat are probably the most well known example of a company that has built a successful business model around supporting and servicing open source software. Reddit are a good example of an organisation who have are open source but are a commercial entity who aren’t selling support and services. In their case the value proposition that they are need to protect is their community and not their code.
Many companies have begun to spin off open source components from their in house software. Apache Cassandra is an example of this: developed at facebook and then released for other people to use and contribute to. In situations like this it can be quite helpful to have another organisation manage your IP and community in order to make sure that the open source project is insulated from the business reality of its original parent company. Facebook doesn't need to run the Cassandra project: its an Apache project and other organisations, such as datastax, can continue to contribute to it.
Another key value add that software foundations bring is being able to offer a neutral home to many projects. Vertx is a great example of this: a while back the excellent technical work being done Tim Fox and friends was overshadowed by Vmware’s decision to try and assert control over the project after Tim Fox moved to Red Hat. This came to a fairly happy resolution when the project moved to the Eclipse foundation.
I'm glad that Apache and Eclipse are doing well. I use a lot of software projects run by them and I feel confident using it professionally when there's a vendor neutral home for this software.
Thanks to Tom Fitzhenry, Chris Sinjakli and John Oliver for reading a draft of this blog post.