AI Prepares for Security Spotlight

There's a lot of talk about the budding application of AI in security. Its potential as a tool to assist analysts is growing, whether they need support refining threat volume, or need to build more advanced predictability or confidence into assessments.

But it can also be a hindrance, is currently immature in the industry, and is over-hyped. Because it's relatively new, it is marketed as a solve-all and that's not helping enterprises define what role or value it has for them. It's time to aim for tangible results from better-qualified investments.

Dustin Hillard, founder and CTO at Versive

"What's basically an intractable problem today is that there is a lot of valuable data out there if [analysts are] actually able to distill it down and comprehend it," Dustin Hillard, founder and CTO at Versive, a firm that uses AI for adversary detection, told Security Now.

Hillard left academia as a highly qualified electrical engineer and in the last decade was a data scientist for Yahoo, thinking up new machine-learning-based advertising relevance and search algorithms, and then a senior scientist for Microsoft, developing Cortana.

He advocates that AI be employed to examine the "big picture" core threat behaviors in the adversary lifecycle, rather than individual data sets. Today's threat analysis tools, which in some cases are similar to ones that hackers use to extract data, are passive and look only for data artifacts.

"AI puts together the human understanding that it takes to create an overall narrative of what's happening, tying together separate events into an overall story," said Hillard. Effectively, AI can act as a filter between more traditional tools that generate vast numbers of alerts, and the analysts, aiming to reduce the number of cases. The idea is to identify issues faster than the average time it takes a hacker can be inside the network -- 100 to 200 days.

Conventional tools create on average 17,000 alerts, but only about 20% are considered reliable and less than 5% investigated (sources "The Cost of Malware Containment," a report conducted by the Ponemon Institute, and "M-trends 2015: A View From the Frontlines,," a report from FireEye). AI can combat these statistics and act as an interpretive system for analyst teams.

But outside of this application in the industry, the utility of AI is in question. In some instances, AI is an overactive brain that actually makes security situations more difficult.

"There are many ways in which AI can make the situation worse, the primary being when anomalies are generated for anomalies' sake," said Hillard, "they're not aligning them and prioritizing them against the true risk."

Also, sometimes, it's difficult to interpret why a result was generated by AI, and analysts don't know if the system is in fact wrong or if it's being incredibly smart and presenting information for a novel reason that's tough to understand. One aspect missing from AI tools is the ability to provide the context of results, making them more easily actionable.

With new AI systems, that represents a steep learning curve in understanding how to use the output to gain an advantage against attackers. Perimeter defense helps, but it's very difficult to defend against a hacker team that is well resourced and has access to novel threat technology, versus enterprise experts who have good knowledge but statistically can't hope to catch everything and plug the gaps.

AI can also help bridge knowledge gaps caused by the current shortage of analysts. According to Versive, the recruitment market is turning into a battleground for experts, resulting in competition between IT security developers and people working in, say, the financial, pharmaceutical or consumer web verticals. "We need automated processing because you'll never be able to hire enough people," said Hillard.

Security professionals are extremely difficult to find and keep. According to Forbes, there will be a global shortage of about 2 million positions within a year, with security analysts one of the most in demand. In the fight for senior security talent, lead engineers can even make more than CISOs (according to Dice, an IT recruitment firm).

Related posts:

— Simon Marshall, Technology Journalist, special to Security Now