The report, Cyber Security Bulletin 2005, was released last week. It claimed that out of 5,198 reported flaws, 812 were found in Microsoft's Windows operating system, 2,328 were found in open-source Unix/Linux systems. The rest were declared to be multiple operating-system vulnerabilities.
The report has attracted criticism from some in the open-source community. Linux vendor Red Hat said the vulnerabilities had been wrongly tagged, and so could not be used to compare the relative security of Windows and Linux/Unix platforms.
"The study is confusing and misleading. When you look at the list, the vulnerabilities are miscategorized," Mark Cox, a consulting software engineer at Red Hat, said. "For example, Firefox is categorized as a Unix/Linux operating-system flaw, but it runs just as well on a Windows platform. Apache and PHP also run just as well on both platforms. There are methodological flaws in the statistics."
In addition, Steven Christey, an editor for Common Vulnerabilities and Exposures, an organization that maintains a common vulnerability database, said that the statistics were no basis for comparison of the relative security of Windows and Linux/Unix, because they had been collected from different sources with different criteria for the collection of flaws.
"In my opinion, refined vulnerability information sources (CVE, Bugtraq, etc.) are still a year or two away from being able to produce comparable statistics," Christey wrote in an open letter posted online.
Secunia's Thomas Kristensen agreed with Christey that the various vulnerability collection sources made comparison more difficult. "I think Steve has got some good points on why comparing vulnerability numbers is difficult," said Kristensen, chief technical officer at the security company.
CERT itself pointed out that the information in its bulletin "should not be considered the result of US-CERT analysis," as it included information from outside sources.
Taking flaw types into account
Secunia thought that the nature of the reported vulnerabilities also made it difficult to compare security on the platforms, as Linux/Unix researchers concentrate on vulnerabilities in local privilege separation, while Windows researchers look at possible remote vulnerabilities.
"Generally, many of the vulnerabilities in Linux/Unix based products are classified as local vulnerabilities, including privilege escalation, local denial of service and local exposure of sensitive data. These kind of vulnerabilities are not regarded as particularly critical, but Linux/Unix researchers tend to focus quite a lot on this category, probably because of Unix's long history of proper privilege separation. This has only recently become more relevant in Windows (NT, 2000, and XP), but many Windows researchers still focus more on remote issues," Secunia said.
The US-CERT study has also caused online debate within the open-source community. In Newsforge, the Linux and open-source online publication, Joe Brockmeier and Joe Barr cast doubt on the vulnerability totals.
"The two figures are not representative of today's two major operating-system platforms. One figure represents the vulnerabilities found in Windows operating systems: XP, NT, 98, and so on. The other represents a total figure not just for Solaris, AIX, HP-UX, the BSDs, and Linux, but for a hundred different versions of Linux," the article said.
Red Hat's Cox said Linux operating systems were more secure for businesses than Windows platforms, as fewer vulnerabilities were critical and patches were brought out more quickly.
"There is also the issue of timing," he said. "With Linux products, critical updates are available within a day. If you look at Red Hat Enterprise Linux 3, the average patch time is under a day. With the recent critical WMF (Windows Meta File) vulnerability, it took Microsoft seven days," he said
Microsoft was not available for comment at the time of writing.
Tom Espiner of ZDNet UK reported from London.
[eom]