Open Source Code Is As Good As Proprietary, Says Coverity

Any conviction that open source software (OSS) is somehow inferior to proprietary code, or vice versa, depending on which side of the development fence you sit, is being dispelled by a report from Coverity.

The company has been scanning millions of lines of open source code for its 2011 Coverity Scan Open Source Integrity Report. The results show that the free code quality is on a par with in-house-developed products.

More thoroughly tested

The company said that this year’s study has been massively upgraded with the introduction of the Coverity 5 development testing platform. The new analysis engine incorporates advances in static analysis to improve results and find more defects in any code under test.

During 2011, the company tested open source projects that totalled over 37 million lines of code and the report also details the results of 300 million lines from anonymous proprietary software produced by Coverity Scan users.

On running the scans, it was found that the average defect density (number of defects per 1,000 lines) for open source was 0.45. In the proprietary code the same scan produced an index of 0.64. In both cases this is better than the 1.0 average defect density measured in commercial software.

The cleanest code was found to be Linux 2.6, PHP 5.3, and PostgreSQL 9.1 which weighed in at 0.62, 0.20 and 0.21 respectively. Coverity said that this recognised superior code quality defines the projects as industry benchmarks.

Rasmus Lerdorf, creator of PHP, said: “The quality of our code is critical to the ongoing success and adoption of PHP, which includes some of the world’s most popular Web sites. As our code grows and becomes more complex, Scan will become even more important for us as a way to help improve our code quality.”

To balance the results, the company compared projects of similar size in the open source and proprietary fields. Choosing codebases of around seven million lines, the defect density was roughly the same at 0.62. The parity is put down to progressive software testing throughout the development process to achieve the best results possible.

During the process, Coverity also gains an insight into application sizes. It found that the average open source project has 832,000 lines of code, while proprietary applications are much larger at 7.5 million lines.

In addition to the new testing software, Coverity has recently appointed Zack Samocha as Coverity’s Scan project director. “The line between open source and proprietary software will continue to blur over time as open source is further cemented in the modern software supply chain,” he said. “Our goal with Scan is to enable more open source projects to adopt development testing as part of their workflow for ongoing quality improvement, as well as further the adoption of open source by providing broader visibility into its quality.”

The report is the result of the largest public/private sector research project on open source software integrity. The project started in 2006, jointly with the US Department of Homeland Security, but is now wholly owned and managed by Coverity.

How well do you know Internet security? Try our quiz and find out!