Over the past year, I have guided a number of different people in my school through the process of evaluating open-source software. Often, we need to compare both open-source and commercial software applications against one another, not an easy task when the development and distribution environments are so different from each other.
With commercial software, the decision-maker usually must evaluate the potential utility of a piece of software without actually being able to use the full version. We are overly reliant on the salesperson, an individual with a vested interest in completing the sale and often incomplete knowledge of the product’s reliability and reputation.
With open-source, there is no salesperson, no automatic advocate for the software. However, one can install and evaluate the actual working software, with some effort. If a test goes very well, you may have already completed much of your implementation process!
Commercial software is normally a finished product that remains static for long periods of time. Open-source is often a work in progress that changes considerably over time. The decision-makers in your organization must understand this difference in order to appreciate the rough edges that often exist with open-source software.
Most free and open-source library systems are extensible and modifiable. During the evaluation process, I often try to excite the review team by making small, easy changes when requested. Change the color scheme. Re-order the elements of an item display. Integrate your organization’s logo into the page template. Display the list of modules available from the developer community. Demonstrate the ability to install one. Demonstrate the interoperability between the FOSS system and other information sources. Draw on the strengths of the open-source model. Demonstrate the vast well of technical support available from the online user community.
Depending on the application, there may exist a recognizable brand name and known reputation. Although open-source library systems have existed for some time, they have not yet achieved the level of brand-name recognition as the better funded open-source successes such as Firefox or Moodle. A brand name can provide a significant amount of comfort to your review team. In fact, some people may have a hard time distinguishing open-source from commercial when the brand is sufficiently well established (see Firefox vs. Internet Explorer).
How long should one evaluate open-source software? Since control over the evaluation process is one advantage of open-source over commercial software, I have sometimes evaluated an open-source solution for up to a year. That is certainly much longer than what you will get with a commercial application! I have also set up multiple FOSS systems simultaneously, which is incredibly handy for the purposes of comparison and evaluation among different open-source options. Sometimes, I have taken the testing process incrementally, dropping some of the contenders when the first tests did not go well and investing further resources into the one or two leading test application. That way, the leading application presents as well as possible, comparing as favorably as possible to the commercial alternatives.
I hope that this provides some useful tips for guiding an open-source evaluation software evaluation process.