ANTenna Blog -- Hardware & Software

What is the most important difference between open-source and proprietary software development? That's a secret -- although not in the way that you might think.

As I mentioned yesterday, many proprietary software vendors believe that secrecy is an important ally in battle to protect the security of their products. When Coverity uses its code-analysis tools to hunt bugs in a proprietary app, for example, the results -- and the response -- are usually kept under wraps.

For the past two years, however, Coverity has offered the use of its code-analysis tools, free of charge, to open-source project maintainers. The results appear on a Coverity-managed site designed specifically for this purpose, along with data that tracks a project's efforts to validate and, if necessary, to fix each defect.

When Coverity launched the site in early 2006, 32 open-source projects had signed on. Today, that initial group has expanded to include 150 open-source apps.

Obviously, these companies don't just refrain from using secrecy as a tool for managing software security issues. They avoid it like the plague.

How does this approach shake out in practice? Check out scan.coverity.com, where Coverity allows open-source maintainers to use its code-analysis technology, free of charge. Within a week of launching scan.coverity.com in March, 2006, developers had fixed more than 900 bugs that inaugural group of 32 open-source apps. Nine months later, developers had fixed more than 6,000 defects.

Those improvements translate to some very tangible benefits for the businesses that rely on some -- or, perhaps, many -- of these open-source products. They will waste less time poring over security advisories and hunting down patches. The will see fewer system crashes and worry less about losing valuable data. And they will dramatically reduce the risk that an unpatched security flaw will roll out the red carpet for unwelcome visitors.

These companies' IT staffers also get the ability to judge software by the numbers, as opposed to getting their "research" from a sales drone with a healthy imagination and a sales quota to fill.

Coverity's code-testing tools, for example, assess the number of unfixed defects per 1,000 lines of source code. How do the open-source projects participating in the scan.coverity.com process rate by this standard? A 2004 Wired.com article, referencing Carnegie Mellon University research, states that the average commercial application has 20 to 30 defects per 1,000 lines of code. Compare that to Coverity's 2006 data, which showed fewer than .5 defects per 1,000 lines in the 32 open-source apps participating at the tiime.

Even that figure, however, doesn't do justice to some of these open-source apps. For instance, AMANDA, a popular open-source data backup system, identified and fixed 128 defects, often within days of discovering them. Today, AMANDA, is an application with nearly 100,000 lines of code -- and a defect rate of zero.

Along with 10 other open-source apps, AMANDA has graduated to "Rung 2" of Coverity's evolving open-source code-assessment system. While it's not clear what, exactly, makes an app worthy of promotion to Rung 2, it looks like a good place to be: Four other Rung 2 apps also have defect rates of zero. Another Rung 2 app, the PHP scripting language, has more than 473,000 lines of code and just five outstanding bugs to fix, giving it a defect rate of .004.

Even the Samba networking protocol -- the weak link in the Rung 2 chain, with 110 outstanding bugs -- has a defect rate of just .018 per thousand lines of code. Given these numbers, that Carnegie Mellon statistic I cited above can be wrong by an order of magnitude . . . and still represent an appalling step down in quality.