Vulnerabilities / Threats
8/21/2013
11:08 AM
Connect Directly
RSS
E-Mail
50%
50%

Microsoft Windows Defender Stumbles In Malware Tests

Microsoft's free anti-virus software came in last among 23 programs at catching known malware in an AV program shootout, says independent testing firm.

9 Android Apps To Improve Security, Privacy
9 Android Apps To Improve Security, Privacy
(click image for larger view)
Microsoft Windows can be secured against new malware threats -- provided users don't rely on the free antivirus software that's included with the operating system.

That's one of the takeaways from an endpoint security study released this week by independent German lab AV-Test.

The researchers evaluated 28 antivirus products against three criteria: protection, repair and usability. Products could earn up to six points in each category for a possible total of 18 points. After tallying the results, AV-Test reported that "Bitdefender, Kaspersky Lab and Symantec lead the field while the protection packages from Avast, F-Secure and GData share fourth place."

Still, all of the products earned top scores of 15.2 points or more -- which stood in sharp contrast to the performance of Microsoft's free offerings. "The test results of these [six] products alone are all nearly five points higher than the overall result obtained by Microsoft's Windows Defender or Security Essentials when used together with the Windows Firewall," reported AV-Test. "This proves that the use of external security solutions can lead to a massive improvement when it comes to system protection."

[ Little Prince George a menace? Read Royal Baby Malware Attacks. ]

Interestingly, the top-ranked applications weren't always the best at stopping malware, as measured by the lab's "protection" tests. "The suites from Bitdefender, F-Secure and Kaspersky all did the best job in this category, achieving detection rates of 100%, while the best free programs, namely those from Avast and AVG, were only able to make it to eighth and twelfth place respectively," said AV-Test. "The Windows Defender provided by Microsoft in its operating system set a very low benchmark value with a detection rate of just 79%."

The protection tests were designed to test each product's real-world detection capabilities, and involved subjecting each product to 400 pieces of brand-new -- aka zero-day -- malware.

The products also were tested using a "reference set" comprising 60,000 pieces of malware. "The malware in the [reference] set is already up to four weeks old," said AV-Test. "Good programs are therefore always able to identify 100% of the malware on this list." Furthermore, AV-Test said products only failed the malware detection test if both their scanner and any additional on-demand detection capabilities couldn't identify the malware. "After all, most of the protection packages not only feature basic detection functions, but also a number of other important tools that they use to identify malware," said the research firm.

Microsoft Windows Defender, however, only detected 97% of the reference set, putting it in last place compared to 23 other products that were also tested three different times in six months. In fact, the only other tested products that failed to achieve a 100% reference-set detection rate were Check Point's ZoneAlarm Free Antivirus and Firewall, and AhnLab's V3 Internet Security, as well as K7's Total Security, although that product was only tested twice between January and June.

Microsoft's free endpoint security software, however, did earn top marks in usability, which only five other tested products managed to equal.

AV-Test also examined the impact of the endpoint security software on system load, and found that malware-stopping power comes at a price: system performance. "Although the best programs in the 'Protection' category also achieved excellent results in this 'System Load' category, none of them were able to score the maximum total of six points," reported AV-Test. "This test category is proof that high security comes at the expense of a certain amount of system performance." On average, the top 10 products earned an average of 4.0 points (out of 6.0) for system load, while the top-ranked product, from Bitdefender, earned 5.2.

Interestingly, AV-Test found variations in the tested programs' effectiveness depending on the version of Windows being used in the test. Overall, the research firm found that on Windows 8, tested antivirus products correctly detected zero-day malware 95% of the time, on average, followed by 93% for Windows XP and 92% for Windows 7.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Thomas Claburn
50%
50%
Thomas Claburn,
User Rank: Moderator
8/22/2013 | 8:14:12 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Indeed, the trouble with studies like this is that they present a "your-mileage-may-vary" situation. If the malware isn't detected but it runs on a system configuration that isn't vulnerable, does it matter?
David F. Carr
50%
50%
David F. Carr,
User Rank: Apprentice
8/22/2013 | 1:14:00 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Ack. I confess I've been trying to save money by telling my kids to ignore the norton free trial on their laptops and use Windows Defender instead.
rradina
50%
50%
rradina,
User Rank: Apprentice
8/22/2013 | 12:49:21 AM
re: Microsoft Windows Defender Stumbles In Malware Tests
In the past MS Defender ranked much more favorably. In fact, it used to be favorable enough that there was little reason to spend money on alternatives for very marginal improvement, if that.

One or more of the following are possibilities:

1) The environment has become much more dangerous
2) MS has let Defender rot
3) Alternatives have made vast improvements
4) The initial test was flawed
5) This latest test is flawed

Still, 97% of four-week old virus isn't bad. If combined with running as a normal user vs. an administrator, Defender still seems to be a reasonable alternative.
Mordock
50%
50%
Mordock,
User Rank: Apprentice
8/21/2013 | 8:16:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Let's not bash MS just because they are MS. But I can tell you that if I were the top guy at Microsoft, I would be demanding a report on my desk within the next 24 hours as to how they missed those items on the reference list and what they were going to do to make sure that never happens again. There is absolutely no excuse for that. Then the report for next Friday will be how are they going to fix the rest of the program. I should hope that they will take this as a major embarrassment.
Michael Endler
50%
50%
Michael Endler,
User Rank: Apprentice
8/21/2013 | 7:53:55 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
I'd like to see some of these tests differentiate the products' success with different categories of malware. There's a ton of malware out there, but some of it is regularly employed in more sophisticated attacks, and some of it is regularly employed in "wide net" attacks that target uninformed users. If a company has done a good job training staff and generally has employees who know not to click suspicious links or open questionable attachments, does it change the way that company would assess one of these products verses another?
vdugasp3c
50%
50%
vdugasp3c,
User Rank: Apprentice
8/21/2013 | 7:48:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
This test is flawed. We run several computer companies with over 30 years experience. One of our companies just does repairs and virus cleaning. When you have around 500 repeat customers coming in once per month for virus cleaning and are all using the same products, but once the products are changed and they begin requiring a virus cleaning once every six months and the worst on our list was Symantec/Norton, Bit Defender and McAfee...someone is paying for high rankings.
DDURBIN1
50%
50%
DDURBIN1,
User Rank: Apprentice
8/21/2013 | 5:33:10 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Is there anything Microsoft does well anymore? When people have a choice, more and more will choose not to use Microsoft.
Register for Dark Reading Newsletters
White Papers
Flash Poll
Current Issue
Cartoon
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2009-5142
Published: 2014-08-21
Cross-site scripting (XSS) vulnerability in timthumb.php in TimThumb 1.09 and earlier, as used in Mimbo Pro 2.3.1 and other products, allows remote attackers to inject arbitrary web script or HTML via the src parameter.

CVE-2010-5302
Published: 2014-08-21
Cross-site scripting (XSS) vulnerability in timthumb.php in TimThumb before 1.15 as of 20100908 (r88), as used in multiple products, allows remote attackers to inject arbitrary web script or HTML via the QUERY_STRING.

CVE-2010-5303
Published: 2014-08-21
Cross-site scripting (XSS) vulnerability in the displayError function in timthumb.php in TimThumb before 1.15 (r85), as used in multiple products, allows remote attackers to inject arbitrary web script or HTML via unspecified vectors related to $errorString.

CVE-2014-0965
Published: 2014-08-21
IBM WebSphere Application Server (WAS) 7.0.x before 7.0.0.33, 8.0.x before 8.0.0.9, and 8.5.x before 8.5.5.3 allows remote attackers to obtain sensitive information via a crafted SOAP response.

CVE-2014-3022
Published: 2014-08-21
IBM WebSphere Application Server (WAS) 7.0.x before 7.0.0.33, 8.0.x before 8.0.0.9, and 8.5.x before 8.5.5.3 allows remote attackers to obtain sensitive information via a crafted URL that triggers an error condition.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Three interviews on critical embedded systems and security, recorded at Black Hat 2014 in Las Vegas.