Chief technology officer of the U.S. based security research and consulting firm Neohapsis, Greg Shiply, told ZDNet Australia most analysts are simply getting it wrong these days.
"While their information may be well backed generally... some of their recent observations on security seem to be driven from angles other that practical experience and know-how," he said.
His sentiments are echoed by home-grown security expert Daniel Lewkovitz, who says enterprises listen to large analysis firms because they don't know any better. "Unlike a doctor or a lawyer, anyone can call themselves a security expert... people will listen to whomever is saying things the loudest."
Shiply says the onus isn't only on the companies providing the data--caveat emptor should apply. He says that people buying a lot of analysis and research material may not have the experience to view that material critically. He says more cooperation is required between peers in the industry. "I'd much rather hear from a chief security officer than an armchair warrior answering the phone all day," he said.
The perception in the marketplace is that decisions backed by analysis put together by a large, established organization are safer from a political stand-point--managers feel more confident in making decisions if they have an analysts' report to fall back on if things go pear-shaped, Shiply says. The only problem is the advice can often be quite bad.
"It's the blind leading the blind," he said.
While not targeting any specific company, Lewkovitz and Shiply also question the independence of analyst firms who take money from the vendors while providing ostensibly independent advice to customers.
Counterpoint: Analysts
On the analysts' side, Gartner's head of research for the Asia Pacific region, Jamie Popkin, strongly rejects the suggestion. "The research is absolutely not affected by what any client pays us... that's the basis of the business that we've had all this time," he told ZDNet Australia.
However, when asked, Popkin would not disclose what proportion of the products and services recommended in the Gartner "magic quadrant" were delivered or designed by companies that are also Gartner clients. "We don't disclose who our clients are or what they pay us."
"The magic quadrant has nothing to do with the client relationship," he added.
Gartner provoked a furor earlier this year when it pronounced Intrusion Detection Systems dead and said clients should move towards Intrusion Prevention Systems.
Security mailing lists erupted, with some saying the company of grotesquely misunderstood security. Martin Roesch, who as the author of the Snort IDS, admits he has a vested interest in its future, made his views known.
"I think Gartner is being inflammatory and creating its own hype cycle," he said in a mailing list posting.
Others say the company's advice was based on negative feedback from clients that don't know how to affectively utilize the technology--one remarked that Gartner was misinterpreting a negative user experience for a technical problem.
"That Gartner report will be distributed to countless journalists," wrote another on the focus-ids mailing list. "These people will be preaching the 'informed' conclusions that Gartner is espousing as gospel."
