Microsoft Office 365 missed 9.3% emails containing spam, phishing, and malware from the beginning of September through early October, report Cyren researchers, who analyzed 10.7 million messages.
The threat intelligence firm gauges clients' email security with its Email Security Gap Analysis tool. Inbound emails are processed by its email security system, and all messages that go on to users' inboxes are BCC'd to Cyren's system for automated analysis.
"It's a standard engagement we have with clients," says Pete Starr, Cyren's director of field engineering. "But occasionally we get some interesting nuggets of information." Researchers were curious about how Office 365 was performing, which led to evaluating its security.
During the month of September, Cyren analyzed 10.7 million emails forwarded by Office 365 to user mailboxes for companies tested during that time frame. Of the messages evaluated, 9.75 million (90.7%) were found to be clean. This included 4.6 million newsletter emails, which made up nearly half of legitimate email traffic.
Nearly one million (9.3% of) messages were spam or malicious emails missed by Office 365, says Cyren, noting that the standard Office365 email service has Exchange Online Protection (EOP) to protect against malware and spam. The "false negatives" should not have made it to inboxes.
Researchers found 957,039 emails, or 8.93% of all email traffic, turned out to be spam. Usually, these messages are filtered out through content scanning or pattern detection applied to elements of the email message or its distribution pattern.
Spam aside, 34,077 emails delivered to Office 365 users were phishing messages. Of these, 18,052 were financial phishing emails requesting banking details or account access, 5,424 were password phishing emails, and 10,601 were general phishing emails.
"The biggest shock was just how much was coming through," says Starr. "Yes, the majority of it is spam, but quite a lot is something you don't want."
He refers to the malware attachments found on 3,900 emails delivered to users. While a tiny percentage (0.04%) of all emails delivered, it's also the most dangerous. Of those malware emails, 1,438 were zero-day attachments with no previously known malware signatures. However, malware attached to 2,462 emails was known and should have been detected.
"What really surprised me was the two-and-a-half thousand samples of known malware," Starr says. "Stuff caught by basic, signature-based detection. You expect that kind of stuff to be filtered out."
Is the customer at fault, or is Microsoft? Starr puts some blame on both parties. "Your average Office 365 customer is less well-configured; they perhaps don’t have the best policies on average," he explains.
However, he continues, Microsoft's solution is particularly reliant on reputation-based filtering, meaning the extent of their knowledge is only as good as their database. Today, with the rise of distributed attacks involving malware, phishing, spam, and botnets, many machines involved are fresh IPs. There's a good chance they won't exist inside an IP reputation database, he says.
"Being able to track new IPs is very, very difficult," says Starr. "You find out about them when it's too late."
For businesses hoping to improve their email security, he advises being more sensible about whitelists, noting that many organizations are too broad when adding domain names to their whitelists and letting potentially harmful messages in.
Another mistake is not appreciating how much valid email exists in other languages, like Chinese or Russian. "People either completely block, or completely allow them," he adds, suggesting users take full advantage of email features to set more specific filters.
Join Dark Reading LIVE for two days of practical cyber defense discussions. Learn from the industry’s most knowledgeable IT security experts. Check out the INsecurity agenda here.