Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

Microsoft Windows Defender Stumbles In Malware Tests

Microsoft's free anti-virus software came in last among 23 programs at catching known malware in an AV program shootout, says independent testing firm.

9 Android Apps To Improve Security, Privacy
9 Android Apps To Improve Security, Privacy
(click image for larger view)
Microsoft Windows can be secured against new malware threats -- provided users don't rely on the free antivirus software that's included with the operating system.

That's one of the takeaways from an endpoint security study released this week by independent German lab AV-Test.

The researchers evaluated 28 antivirus products against three criteria: protection, repair and usability. Products could earn up to six points in each category for a possible total of 18 points. After tallying the results, AV-Test reported that "Bitdefender, Kaspersky Lab and Symantec lead the field while the protection packages from Avast, F-Secure and GData share fourth place."

Still, all of the products earned top scores of 15.2 points or more -- which stood in sharp contrast to the performance of Microsoft's free offerings. "The test results of these [six] products alone are all nearly five points higher than the overall result obtained by Microsoft's Windows Defender or Security Essentials when used together with the Windows Firewall," reported AV-Test. "This proves that the use of external security solutions can lead to a massive improvement when it comes to system protection."

[ Little Prince George a menace? Read Royal Baby Malware Attacks. ]

Interestingly, the top-ranked applications weren't always the best at stopping malware, as measured by the lab's "protection" tests. "The suites from Bitdefender, F-Secure and Kaspersky all did the best job in this category, achieving detection rates of 100%, while the best free programs, namely those from Avast and AVG, were only able to make it to eighth and twelfth place respectively," said AV-Test. "The Windows Defender provided by Microsoft in its operating system set a very low benchmark value with a detection rate of just 79%."

The protection tests were designed to test each product's real-world detection capabilities, and involved subjecting each product to 400 pieces of brand-new -- aka zero-day -- malware.

The products also were tested using a "reference set" comprising 60,000 pieces of malware. "The malware in the [reference] set is already up to four weeks old," said AV-Test. "Good programs are therefore always able to identify 100% of the malware on this list." Furthermore, AV-Test said products only failed the malware detection test if both their scanner and any additional on-demand detection capabilities couldn't identify the malware. "After all, most of the protection packages not only feature basic detection functions, but also a number of other important tools that they use to identify malware," said the research firm.

Microsoft Windows Defender, however, only detected 97% of the reference set, putting it in last place compared to 23 other products that were also tested three different times in six months. In fact, the only other tested products that failed to achieve a 100% reference-set detection rate were Check Point's ZoneAlarm Free Antivirus and Firewall, and AhnLab's V3 Internet Security, as well as K7's Total Security, although that product was only tested twice between January and June.

Microsoft's free endpoint security software, however, did earn top marks in usability, which only five other tested products managed to equal.

AV-Test also examined the impact of the endpoint security software on system load, and found that malware-stopping power comes at a price: system performance. "Although the best programs in the 'Protection' category also achieved excellent results in this 'System Load' category, none of them were able to score the maximum total of six points," reported AV-Test. "This test category is proof that high security comes at the expense of a certain amount of system performance." On average, the top 10 products earned an average of 4.0 points (out of 6.0) for system load, while the top-ranked product, from Bitdefender, earned 5.2.

Interestingly, AV-Test found variations in the tested programs' effectiveness depending on the version of Windows being used in the test. Overall, the research firm found that on Windows 8, tested antivirus products correctly detected zero-day malware 95% of the time, on average, followed by 93% for Windows XP and 92% for Windows 7.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Thomas Claburn
Thomas Claburn,
User Rank: Ninja
8/22/2013 | 8:14:12 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Indeed, the trouble with studies like this is that they present a "your-mileage-may-vary" situation. If the malware isn't detected but it runs on a system configuration that isn't vulnerable, does it matter?
David F. Carr
David F. Carr,
User Rank: Apprentice
8/22/2013 | 1:14:00 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Ack. I confess I've been trying to save money by telling my kids to ignore the norton free trial on their laptops and use Windows Defender instead.
User Rank: Apprentice
8/22/2013 | 12:49:21 AM
re: Microsoft Windows Defender Stumbles In Malware Tests
In the past MS Defender ranked much more favorably. In fact, it used to be favorable enough that there was little reason to spend money on alternatives for very marginal improvement, if that.

One or more of the following are possibilities:

1) The environment has become much more dangerous
2) MS has let Defender rot
3) Alternatives have made vast improvements
4) The initial test was flawed
5) This latest test is flawed

Still, 97% of four-week old virus isn't bad. If combined with running as a normal user vs. an administrator, Defender still seems to be a reasonable alternative.
User Rank: Apprentice
8/21/2013 | 8:16:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Let's not bash MS just because they are MS. But I can tell you that if I were the top guy at Microsoft, I would be demanding a report on my desk within the next 24 hours as to how they missed those items on the reference list and what they were going to do to make sure that never happens again. There is absolutely no excuse for that. Then the report for next Friday will be how are they going to fix the rest of the program. I should hope that they will take this as a major embarrassment.
Michael Endler
Michael Endler,
User Rank: Apprentice
8/21/2013 | 7:53:55 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
I'd like to see some of these tests differentiate the products' success with different categories of malware. There's a ton of malware out there, but some of it is regularly employed in more sophisticated attacks, and some of it is regularly employed in "wide net" attacks that target uninformed users. If a company has done a good job training staff and generally has employees who know not to click suspicious links or open questionable attachments, does it change the way that company would assess one of these products verses another?
User Rank: Apprentice
8/21/2013 | 7:48:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
This test is flawed. We run several computer companies with over 30 years experience. One of our companies just does repairs and virus cleaning. When you have around 500 repeat customers coming in once per month for virus cleaning and are all using the same products, but once the products are changed and they begin requiring a virus cleaning once every six months and the worst on our list was Symantec/Norton, Bit Defender and McAfee...someone is paying for high rankings.
User Rank: Apprentice
8/21/2013 | 5:33:10 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Is there anything Microsoft does well anymore? When people have a choice, more and more will choose not to use Microsoft.
4 Tips to Run Fast in the Face of Digital Transformation
Shane Buckley, President & Chief Operating Officer, Gigamon,  12/9/2019
US Sets $5 Million Bounty For Russian Hacker Behind Zeus Banking Thefts
Jai Vijayan, Contributing Writer,  12/5/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: Our Endpoint Protection system is a little outdated... 
Current Issue
The Year in Security: 2019
This Tech Digest provides a wrap up and overview of the year's top cybersecurity news stories. It was a year of new twists on old threats, with fears of another WannaCry-type worm and of a possible botnet army of Wi-Fi routers. But 2019 also underscored the risk of firmware and trusted security tools harboring dangerous holes that cybercriminals and nation-state hackers could readily abuse. Read more.
Flash Poll
Rethinking Enterprise Data Defense
Rethinking Enterprise Data Defense
Frustrated with recurring intrusions and breaches, cybersecurity professionals are questioning some of the industrys conventional wisdom. Heres a look at what theyre thinking about.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-12-11
Orca has arbitrary code execution due to insecure Python module load
PUBLISHED: 2019-12-11
RubyGem omniauth-facebook has an access token security vulnerability
PUBLISHED: 2019-12-11
JBossWeb Bayeux has reflected XSS
PUBLISHED: 2019-12-11
node-connect before 2.8.2 has cross site scripting in methodOverride Middleware
PUBLISHED: 2019-12-11
Progress Telerik UI for ASP.NET AJAX through 2019.3.1023 contains a .NET deserialization vulnerability in the RadAsyncUpload function. This is exploitable when the encryption keys are known due to the presence of CVE-2017-11317 or CVE-2017-11357, or other means. Exploitation can result in remote cod...