Vulnerabilities / Threats
11:08 AM

Microsoft Windows Defender Stumbles In Malware Tests

Microsoft's free anti-virus software came in last among 23 programs at catching known malware in an AV program shootout, says independent testing firm.

9 Android Apps To Improve Security, Privacy
9 Android Apps To Improve Security, Privacy
(click image for larger view)
Microsoft Windows can be secured against new malware threats -- provided users don't rely on the free antivirus software that's included with the operating system.

That's one of the takeaways from an endpoint security study released this week by independent German lab AV-Test.

The researchers evaluated 28 antivirus products against three criteria: protection, repair and usability. Products could earn up to six points in each category for a possible total of 18 points. After tallying the results, AV-Test reported that "Bitdefender, Kaspersky Lab and Symantec lead the field while the protection packages from Avast, F-Secure and GData share fourth place."

Still, all of the products earned top scores of 15.2 points or more -- which stood in sharp contrast to the performance of Microsoft's free offerings. "The test results of these [six] products alone are all nearly five points higher than the overall result obtained by Microsoft's Windows Defender or Security Essentials when used together with the Windows Firewall," reported AV-Test. "This proves that the use of external security solutions can lead to a massive improvement when it comes to system protection."

[ Little Prince George a menace? Read Royal Baby Malware Attacks. ]

Interestingly, the top-ranked applications weren't always the best at stopping malware, as measured by the lab's "protection" tests. "The suites from Bitdefender, F-Secure and Kaspersky all did the best job in this category, achieving detection rates of 100%, while the best free programs, namely those from Avast and AVG, were only able to make it to eighth and twelfth place respectively," said AV-Test. "The Windows Defender provided by Microsoft in its operating system set a very low benchmark value with a detection rate of just 79%."

The protection tests were designed to test each product's real-world detection capabilities, and involved subjecting each product to 400 pieces of brand-new -- aka zero-day -- malware.

The products also were tested using a "reference set" comprising 60,000 pieces of malware. "The malware in the [reference] set is already up to four weeks old," said AV-Test. "Good programs are therefore always able to identify 100% of the malware on this list." Furthermore, AV-Test said products only failed the malware detection test if both their scanner and any additional on-demand detection capabilities couldn't identify the malware. "After all, most of the protection packages not only feature basic detection functions, but also a number of other important tools that they use to identify malware," said the research firm.

Microsoft Windows Defender, however, only detected 97% of the reference set, putting it in last place compared to 23 other products that were also tested three different times in six months. In fact, the only other tested products that failed to achieve a 100% reference-set detection rate were Check Point's ZoneAlarm Free Antivirus and Firewall, and AhnLab's V3 Internet Security, as well as K7's Total Security, although that product was only tested twice between January and June.

Microsoft's free endpoint security software, however, did earn top marks in usability, which only five other tested products managed to equal.

AV-Test also examined the impact of the endpoint security software on system load, and found that malware-stopping power comes at a price: system performance. "Although the best programs in the 'Protection' category also achieved excellent results in this 'System Load' category, none of them were able to score the maximum total of six points," reported AV-Test. "This test category is proof that high security comes at the expense of a certain amount of system performance." On average, the top 10 products earned an average of 4.0 points (out of 6.0) for system load, while the top-ranked product, from Bitdefender, earned 5.2.

Interestingly, AV-Test found variations in the tested programs' effectiveness depending on the version of Windows being used in the test. Overall, the research firm found that on Windows 8, tested antivirus products correctly detected zero-day malware 95% of the time, on average, followed by 93% for Windows XP and 92% for Windows 7.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Thomas Claburn
Thomas Claburn,
User Rank: Ninja
8/22/2013 | 8:14:12 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Indeed, the trouble with studies like this is that they present a "your-mileage-may-vary" situation. If the malware isn't detected but it runs on a system configuration that isn't vulnerable, does it matter?
David F. Carr
David F. Carr,
User Rank: Apprentice
8/22/2013 | 1:14:00 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Ack. I confess I've been trying to save money by telling my kids to ignore the norton free trial on their laptops and use Windows Defender instead.
User Rank: Apprentice
8/22/2013 | 12:49:21 AM
re: Microsoft Windows Defender Stumbles In Malware Tests
In the past MS Defender ranked much more favorably. In fact, it used to be favorable enough that there was little reason to spend money on alternatives for very marginal improvement, if that.

One or more of the following are possibilities:

1) The environment has become much more dangerous
2) MS has let Defender rot
3) Alternatives have made vast improvements
4) The initial test was flawed
5) This latest test is flawed

Still, 97% of four-week old virus isn't bad. If combined with running as a normal user vs. an administrator, Defender still seems to be a reasonable alternative.
User Rank: Apprentice
8/21/2013 | 8:16:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Let's not bash MS just because they are MS. But I can tell you that if I were the top guy at Microsoft, I would be demanding a report on my desk within the next 24 hours as to how they missed those items on the reference list and what they were going to do to make sure that never happens again. There is absolutely no excuse for that. Then the report for next Friday will be how are they going to fix the rest of the program. I should hope that they will take this as a major embarrassment.
Michael Endler
Michael Endler,
User Rank: Apprentice
8/21/2013 | 7:53:55 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
I'd like to see some of these tests differentiate the products' success with different categories of malware. There's a ton of malware out there, but some of it is regularly employed in more sophisticated attacks, and some of it is regularly employed in "wide net" attacks that target uninformed users. If a company has done a good job training staff and generally has employees who know not to click suspicious links or open questionable attachments, does it change the way that company would assess one of these products verses another?
User Rank: Apprentice
8/21/2013 | 7:48:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
This test is flawed. We run several computer companies with over 30 years experience. One of our companies just does repairs and virus cleaning. When you have around 500 repeat customers coming in once per month for virus cleaning and are all using the same products, but once the products are changed and they begin requiring a virus cleaning once every six months and the worst on our list was Symantec/Norton, Bit Defender and McAfee...someone is paying for high rankings.
User Rank: Apprentice
8/21/2013 | 5:33:10 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Is there anything Microsoft does well anymore? When people have a choice, more and more will choose not to use Microsoft.
Register for Dark Reading Newsletters
White Papers
Current Issue
Dark Reading Tech Digest September 7, 2015
Some security flaws go beyond simple app vulnerabilities. Have you checked for these?
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2015-10-09
Simple Streams (simplestreams) does not properly verify the GPG signatures of disk image files, which allows remote mirror servers to spoof disk images and have unspecified other impact via a 403 (aka Forbidden) response.

Published: 2015-10-09
The Telephony component in Apple OS X before 10.11, when the Continuity feature is enabled, allows local users to bypass intended telephone-call restrictions via unspecified vectors.

Published: 2015-10-09
IcedTea-Web before 1.5.3 and 1.6.x before 1.6.1 does not properly sanitize applet URLs, which allows remote attackers to inject applets into the .appletTrustSettings configuration file and bypass user approval to execute the applet via a crafted web page, possibly related to line breaks.

Published: 2015-10-09
IcedTea-Web before 1.5.3 and 1.6.x before 1.6.1 does not properly determine the origin of unsigned applets, which allows remote attackers to bypass the approval process or trick users into approving applet execution via a crafted web page.

Published: 2015-10-09
The Safari Extensions implementation in Apple Safari before 9 does not require user confirmation before replacing an installed extension, which has unspecified impact and attack vectors.

Dark Reading Radio
Archived Dark Reading Radio
What can the information security industry do to solve the IoT security problem? Learn more and join the conversation on the next episode of Dark Reading Radio.