Vulnerabilities / Threats

11:08 AM

Microsoft Windows Defender Stumbles In Malware Tests

Microsoft's free anti-virus software came in last among 23 programs at catching known malware in an AV program shootout, says independent testing firm.

9 Android Apps To Improve Security, Privacy
9 Android Apps To Improve Security, Privacy
(click image for larger view)
Microsoft Windows can be secured against new malware threats -- provided users don't rely on the free antivirus software that's included with the operating system.

That's one of the takeaways from an endpoint security study released this week by independent German lab AV-Test.

The researchers evaluated 28 antivirus products against three criteria: protection, repair and usability. Products could earn up to six points in each category for a possible total of 18 points. After tallying the results, AV-Test reported that "Bitdefender, Kaspersky Lab and Symantec lead the field while the protection packages from Avast, F-Secure and GData share fourth place."

Still, all of the products earned top scores of 15.2 points or more -- which stood in sharp contrast to the performance of Microsoft's free offerings. "The test results of these [six] products alone are all nearly five points higher than the overall result obtained by Microsoft's Windows Defender or Security Essentials when used together with the Windows Firewall," reported AV-Test. "This proves that the use of external security solutions can lead to a massive improvement when it comes to system protection."

[ Little Prince George a menace? Read Royal Baby Malware Attacks. ]

Interestingly, the top-ranked applications weren't always the best at stopping malware, as measured by the lab's "protection" tests. "The suites from Bitdefender, F-Secure and Kaspersky all did the best job in this category, achieving detection rates of 100%, while the best free programs, namely those from Avast and AVG, were only able to make it to eighth and twelfth place respectively," said AV-Test. "The Windows Defender provided by Microsoft in its operating system set a very low benchmark value with a detection rate of just 79%."

The protection tests were designed to test each product's real-world detection capabilities, and involved subjecting each product to 400 pieces of brand-new -- aka zero-day -- malware.

The products also were tested using a "reference set" comprising 60,000 pieces of malware. "The malware in the [reference] set is already up to four weeks old," said AV-Test. "Good programs are therefore always able to identify 100% of the malware on this list." Furthermore, AV-Test said products only failed the malware detection test if both their scanner and any additional on-demand detection capabilities couldn't identify the malware. "After all, most of the protection packages not only feature basic detection functions, but also a number of other important tools that they use to identify malware," said the research firm.

Microsoft Windows Defender, however, only detected 97% of the reference set, putting it in last place compared to 23 other products that were also tested three different times in six months. In fact, the only other tested products that failed to achieve a 100% reference-set detection rate were Check Point's ZoneAlarm Free Antivirus and Firewall, and AhnLab's V3 Internet Security, as well as K7's Total Security, although that product was only tested twice between January and June.

Microsoft's free endpoint security software, however, did earn top marks in usability, which only five other tested products managed to equal.

AV-Test also examined the impact of the endpoint security software on system load, and found that malware-stopping power comes at a price: system performance. "Although the best programs in the 'Protection' category also achieved excellent results in this 'System Load' category, none of them were able to score the maximum total of six points," reported AV-Test. "This test category is proof that high security comes at the expense of a certain amount of system performance." On average, the top 10 products earned an average of 4.0 points (out of 6.0) for system load, while the top-ranked product, from Bitdefender, earned 5.2.

Interestingly, AV-Test found variations in the tested programs' effectiveness depending on the version of Windows being used in the test. Overall, the research firm found that on Windows 8, tested antivirus products correctly detected zero-day malware 95% of the time, on average, followed by 93% for Windows XP and 92% for Windows 7.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Thomas Claburn
Thomas Claburn,
User Rank: Ninja
8/22/2013 | 8:14:12 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Indeed, the trouble with studies like this is that they present a "your-mileage-may-vary" situation. If the malware isn't detected but it runs on a system configuration that isn't vulnerable, does it matter?
David F. Carr
David F. Carr,
User Rank: Apprentice
8/22/2013 | 1:14:00 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Ack. I confess I've been trying to save money by telling my kids to ignore the norton free trial on their laptops and use Windows Defender instead.
User Rank: Apprentice
8/22/2013 | 12:49:21 AM
re: Microsoft Windows Defender Stumbles In Malware Tests
In the past MS Defender ranked much more favorably. In fact, it used to be favorable enough that there was little reason to spend money on alternatives for very marginal improvement, if that.

One or more of the following are possibilities:

1) The environment has become much more dangerous
2) MS has let Defender rot
3) Alternatives have made vast improvements
4) The initial test was flawed
5) This latest test is flawed

Still, 97% of four-week old virus isn't bad. If combined with running as a normal user vs. an administrator, Defender still seems to be a reasonable alternative.
User Rank: Apprentice
8/21/2013 | 8:16:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Let's not bash MS just because they are MS. But I can tell you that if I were the top guy at Microsoft, I would be demanding a report on my desk within the next 24 hours as to how they missed those items on the reference list and what they were going to do to make sure that never happens again. There is absolutely no excuse for that. Then the report for next Friday will be how are they going to fix the rest of the program. I should hope that they will take this as a major embarrassment.
Michael Endler
Michael Endler,
User Rank: Apprentice
8/21/2013 | 7:53:55 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
I'd like to see some of these tests differentiate the products' success with different categories of malware. There's a ton of malware out there, but some of it is regularly employed in more sophisticated attacks, and some of it is regularly employed in "wide net" attacks that target uninformed users. If a company has done a good job training staff and generally has employees who know not to click suspicious links or open questionable attachments, does it change the way that company would assess one of these products verses another?
User Rank: Apprentice
8/21/2013 | 7:48:50 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
This test is flawed. We run several computer companies with over 30 years experience. One of our companies just does repairs and virus cleaning. When you have around 500 repeat customers coming in once per month for virus cleaning and are all using the same products, but once the products are changed and they begin requiring a virus cleaning once every six months and the worst on our list was Symantec/Norton, Bit Defender and McAfee...someone is paying for high rankings.
User Rank: Apprentice
8/21/2013 | 5:33:10 PM
re: Microsoft Windows Defender Stumbles In Malware Tests
Is there anything Microsoft does well anymore? When people have a choice, more and more will choose not to use Microsoft.
Want Your Daughter to Succeed in Cyber? Call Her John
John De Santis, CEO, HyTrust,  5/16/2018
Don't Roll the Dice When Prioritizing Vulnerability Fixes
Ericka Chickowski, Contributing Writer, Dark Reading,  5/15/2018
New Mexico Man Sentenced on DDoS, Gun Charges
Dark Reading Staff 5/18/2018
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: "Security through obscurity"
Current Issue
Flash Poll
[Strategic Security Report] Navigating the Threat Intelligence Maze
[Strategic Security Report] Navigating the Threat Intelligence Maze
Most enterprises are using threat intel services, but many are still figuring out how to use the data they're collecting. In this Dark Reading survey we give you a look at what they're doing today - and where they hope to go.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2018-05-21
jenkins before versions 2.44, 2.32.2 is vulnerable to a persisted cross-site scripting vulnerability in console notes (SECURITY-382). Jenkins allows plugins to annotate build logs, adding new content or changing the presentation of existing content while the build is running. Malicious Jenkins users...
PUBLISHED: 2018-05-21
kernel drivers before version 4.17-rc1 are vulnerable to a weakness in the Linux kernel's implementation of random seed data. Programs, early in the boot sequence, could use the data allocated for the seed before it was sufficiently generated.
PUBLISHED: 2018-05-21
An issue was discovered in Pluck before 4.7.6. There is authenticated stored XSS because the character set for filenames is not properly restricted.
PUBLISHED: 2018-05-21
An issue was discovered in Pluck before 4.7.6. Remote PHP code execution is possible because the set of disallowed filetypes for uploads in missing some applicable ones such as .phtml and .htaccess.
PUBLISHED: 2018-05-21
The Micro Focus Client for OES before version 2 SP4 IR8a has a vulnerability that could allow a local attacker to elevate privileges via a buffer overflow in ncfsd.sys.