Over the last couple of weeks, Wikipedia, the free, open-access encyclopedia, has taken a great deal of flak in the press for problems related to the credibility of its authors and its general accountability.
In particular, Wikipedia has taken hits for its inclusion, for four months, of an anonymously written article linking former journalist John Seigenthaler to the assassinations of Robert Kennedy and John F. Kennedy. At the same time, the blogosphere was buzzing for several days about podcasting pioneer Adam Curry's being accused of anonymously deleting references to others' seminal work on the technology.
Growing pains for Wikipedia
In response to situations like these and others in its history, Wikipedia founder Jimmy Wales has always maintained that the service and its community are built around a self-policing and self-cleaning nature that is supposed to ensure its articles are accurate.
Still, many critics have tried to downplay its role as a source of valid information and have often pointed to the Encyclopedia Britannica as an example of an accurate reference.
For its study, Nature chose articles from both sites in a wide range of topics and sent them to what it called "relevant" field experts for peer review. The experts then compared the competing articles--one from each site on a given topic--side by side, but were not told which article came from which site. Nature got back 42 usable reviews from its field of experts.
In the end, the journal found just eight serious errors, such as general misunderstandings of vital concepts, in the articles. Of those, four came from each site. They did, however, discover a series of factual errors, omissions or misleading statements. All told, Wikipedia had 162 such problems, while Britannica had 123.
That averages out to 2.92 mistakes per article for Britannica and 3.86 for Wikipedia.
"An expert-led investigation carried out by Nature--the first to use peer review to compare Wikipedia and Britannica's coverage of science," the journal wrote, "suggests that such high-profile examples (like the Seigenthaler and Curry situations) are the exception rather than the rule."
And to Wales, while Britannica came out looking a little bit more accurate than Wikipedia, the Nature study was validation of his service's fundamental structure.
"I was very pleased, just to see that (the study) was reasonably favorable," Wales told CNET News.com. "I think it provides, for us, a great counterpoint to the press coverage we've gotten recently, because it puts the focus on the broader quality and not just one article."
He also acknowledged that the error rate for each encyclopedia was not insignificant, and added that he thinks such numbers demonstrate that broad review of encyclopedia articles is needed.
He also said that the results belie the notion that Britannica is infallible.
"I have very great respect for Britannica," Wales said. But "I think there is a general view among a lot of people that it has no errors, like, 'I read it in Britannica, it must be true.' It's good that people see that there are a lot of errors everywhere."
To Britannica officials, however, the Nature results showed that Wikipedia still has a way to go.
"The (Nature) article is saying that Wikipedia has a third more errors" than Britannica, said Jorge Cauz, president of Encyclopedia Britannica.
But Cauz and editor in chief Dale Hoiberg also said they were concerned that Nature had not specified the problems that it had found in Britannica.
"We've asked them a number of questions about the process they used," Hoiberg said. "They said in (their article) that the inaccuracies included errors, omissions and misleading statements. But there's no indication of how many of each. So we're very eager to look at that and explore it because we take it very seriously."
