Having a lot of open source to choose from is definitely a good thing. However, the downside is that you have to do some actual work to figure out which is best for your project. So here’s are some rules of thumb on how to quickly evaluate and pick an open source library. Specific emphasis on “library” since our intent is to incorporate it into our own work as opposed to simple usage.
- Usage – Google’s PageRank algortithm ranks web pages based on the quantity and quality of other web pages linking to it. It’s similar to the rule of thumb of looking at citations to discern the quality of an academic paper. In similar spirit an open source library’s quality can be discerned from the number and quality of other open and closed source projects that use it. For example, the Hibernate project has a ton of projects that not only use it but provide tools to support it.
- Extensibility – An essential quality of most popular projects is the presence of a well defined mechanism for extension. Look for this in the library your are evaluating, the lack of this quality can put you in a bind that’s extremely difficult to extricate oneself from. For example, Eclipse and JEdit have well defined plugin architectures that make it workable for multiple contributors to enhance the platform without stepping on one another.
- Velocity – Is the project improving on a consistent and steady pace? Some well known open source projects (i.e. JDOM) have been progressing at a glacial pace, this observation along should make you turn to a more vibrant project like DOM4J. JDOM is an example of a project that has fallen victim to its premature popularity.
- Scafolding – You should seriously be critical of a project if it lacks an easy way to build.
The lack of a good build system implies a lack of support for collaborative development. In otherwords, the originators don’t find it necessary because its most likely nobody else is collaborating with them.
- API Usability Testing – Now it should be obvious that any good software project requires extensive testing. However, the presence of extensive unit tests indicates other qualities. That is, it indicates a discipline of supporting iterative development. Good projects are built in iterative fashion, this implies a need to regression test against previous iterations and this also implies experimentation of the form of APIs. APIs just like user interfaces require an iterative approach to implement well, if you can’t see any indication of such an approach being applied to APIs then expect to see APIs that feel like UIs that forgot to do any usability testing.
- How To’s – Information on the kind of how to use this stuff or even better if its available as unit tests. Ignore “marketecture”, that is schpiels about themselves, rather look for evidence like blogs, external articles or third party books that describes actual usage of the library. If someone else has invested the time to write about how they leveraged the tool then it contributes some credibility. In addition, you get yourself up and running faster.
- Developed NOT in a Vacuum – The problem with all too many libraries is that they’re developed in a vacuum. Look for indicators that the library is developed as part of a much larger project. This indicates that most of the functionality is added because of an actual requirement as opposed to some developer’s whim. The last thing we want to use is a library with feature creep.
- Attention to Detail – When you see indications of attention to the minutae then you may have struck gold. For example, LOG4J takes great pains fine tuning the time it takes to log events as opposed to simply provifing a logging framework. The design of LOG4J is a consequence of these painstaking efforts and not out of the collective wisdom (i.e. least common denominator) of a commitee.
- Sponsorship – Believe it or not but many of the best open source projects come from paid efforts. The good ones usually come out of either from a corporate’s internal project, a result of a consulting engagement or from a research group. The better ones come from a continuous and stead stream of sponsorship. Money doesn’t buy you quality, however money does buy you continuous and steady development. Remember, boredom can kill even the best projects, however cash is a good distraction from monotony.
- Don’t forget the Code -
What about the code? If you do have the time, then do look at the code, afterall, that’s were the rubber meets the road. You can run the code against many of the static code checkers that are available for Java. They’ll give you a quick sample of the quality of the underlying code. It’s not going to tell you if it’s any good, however, it’ll at least weed out the ugly.
Remember also to consider suitability. That is are you comfortable with level of standards compliance, the licensing restrictions and availability of support? The best technical solution is not necessarily the best for your needs.
Finally, in no way should the choice to use open source interfere with the success of your projects. The main benefit of open source is options, rather than the traditional “buy vs build?” question, it is now more like “buy vs build vs borrow?”