SAN FRANCISCO -- RSA 2008 Conference -- Fortinet emerged as the winner of an event that was part independent product testing, part porn-filter bakeoff.

Wednesday's "Deep Throat Fight Club" event, sponsored by Untangle, pitted five different vendors against one another to filter out the naughty, the explicit, and the grossly anatomical. (See 'Fight Club' Aims to Test Pornography Filters.) In addition to Fortinet, the comparison testing included Barracuda, SonicWall, Watchguard, and WebSense. ScanSafe was part of the original lineup of vendors, but as a hosted solution, the vendor couldn't reconfigure properly in time for the test, the organizers said.

Untangle undertook the test because its own customers complained that products in place were missing sites and that there were too many ways to circumvent the filters. Access to potentially objectionable sites creates legal, moral, and even security issues, since porn sites are increasingly used as vehicles for the propagation of all kinds of malware, the organization said. The event was also a follow-on to a similar antivirus test last summer. (See Antivirus Tools Underperform When Tested in LinuxWorld 'Fight Club'.)

Untangle had each filter scan some 5,000 URLs, using key words and phrases like "nudism," "pornography," "adult/mature content," "intimate apparel/swimsuit," and "not sex education." Each filter was then graded for the percentage of sites correctly blocked, the percentage of sites it missed, and the number of false positives.

Untangle was also careful to emphasize what its testing did not cover -- like performance and speed, IP filtering ability, and other categories like proxies. It also didn't examine or compare advanced features like SSL scanning or safe searches, according to Dirk Morris, CTO and co-founder of Untangle.

Fortinet came out on top, having correctly identified 98 percent of the porn sites in the sample; Barracuda posted the least impressive results with 94 percent. Watchguard and WebSense tied right behind Fortinet; SonicWall was next, followed by Barracuda. Of the five vendors, SonicWall had the most false positives, while Barracuda missed the greatest number of porn sites in its filtering.

Untangle representatives said they were surprised by the number of false positives across the board, and Morris used the event to remind customers to be sure to evaluate such content filters on a broad set of criteria, including price, packaging, and support.

"All products not only missed porn but suffered from false positives, which can be costly to businesses using web filters," Morris said, in a statement. "For some businesses these will make great solutions, but for businesses and schools expecting to block all porn this approach won't cover it all."

The models tested include Barracuda's Web Filter 210 with 3.2.1.021 (2008-01-18) firmware and 1.0.676 (2008-04-08) database; Fortinet's 50A with 3.00-b0662 (MR6 Patch 1) firmware; SonicWall's Pro1260 with SonicOS Enhanced 3.2.3.0-6e; Watchguard's x20e, with build 10.0.2 - Mar 1 2008 and 171410 firmware; and WebSense Express 1.0 with build 20070611_5114 and database 23020 (2008-04-08).

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.