Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

4/29/2021
01:00 PM
50%
50%

Adobe Open Sources Tool for Anomaly Research

The One-Stop Anomaly Shop (OSAS) project packages machine-learning algorithms into a Docker container for finding anomalies in security log data.

Adobe has released an open source project to detect and classify anomalies in security log data using a tool the company says is simple to run and easily modified. 

The One-Stop Anomaly Shop (OSAS) is an open source machine-learning (ML) tool that can add structure to log data by generating labels for different types of data and then use that data as the inputs to classification algorithms. The approach solves several ML problems — such as data sparsity and overfitting — while giving security analysts a macro view of log data that allows easier analysis, states Adobe's security intelligence team in a technical paper on their approach.

Related Content:

Microsoft Uses Machine Learning to Predict Attackers' Next Steps

Special Report: Assessing Cybersecurity Risk in Today's Enterprises

New From The Edge: 10K Hackers Defend the Planet Against Extraterrestrials

The software project allows security teams to quickly identify what features — or feature combinations — provide the most benefit in terms of analysis in a particular dataset, the Adobe Security Intelligence Team told Dark Reading in an email interview. 

"One could consider finding anomalies as somewhat trivial from the computational perspective," they said. "However, being able to say why something is an anomaly is an entirely different story. OSAS is useful in identifying why [an event is considered] an anomaly."

The One-Stop Anomaly Shop project provides security analysts and researchers with a way to quickly analyze security logs using labeled data, even when the security log file has a variety of event types, the Adobe group stated in a blog post. The project, available on GitHub, creates a Docker container running the Elasticsearch search engine, Logstash indexer, and Kibana Web front-end — a combination known as the ELK stack — while the ML application is written in Python.

The system labels events with a variety of tags, indicating, for example, whether the anomaly is unique, whether a particular port, process, or path is rare, and whether the event connects to a public IP or the localhost. 

"There is a lot of research and whitepapers on data-science in security, but few tools that implement state-of-the-art ideas that are made available to the community," Adobe's team told Dark Reading. "Primarily, open-sourcing OSAS was an opportunity for us to put our work in an end-to-end framework. Secondly, we want to make OSAS as robust and security oriented as possible and we cannot achieve that without support from the security community."

Adobe is not the only software company to provide security teams with ML tools. Earlier this month, Microsoft published details of a project that uses the company's massive data set of attack traffic, along with the MITRE ATT&CK framework, to build an ML model that not only assigns particular attack tactics with certain groups, but predicts the attacker's potential next steps.

Adobe's tool is best-suited for working with security log data, but it can work on any source of flat log data that follow the same patterns, such as authentication logs, Web server logs, and access logs, Adobe stated.

After tagging the various elements of the log file, a second pipeline also assigns risk-based scores to collections of tags.

"The primary goal is to assign high scores to suspicious activity and low scores to normal operations," the security intelligence team states in its technical whitepaper.

The ML tool has multiple strategies for detecting anomalies in log files and assigning them risk-based scores. An unsupervised-learning approach may find malicious activities that would go undiscovered with supervised-learning models but will also likely generate more false alerts. 

"In theory, potentially malicious events are a subset of the anomalies set," the security intelligence team said via email. "Targeting the detection of these potentially malicious events by OSAS can be achieved by creating a tailored data-grooming pipeline via the configuration file."

In tests, the security intelligence team used benign data of normal operations to train the ML algorithm and then input an artificially constructed dataset to benchmark the ability to detect anomalies. The supervised approach had a nearly 95% detection rate, while two unsupervised models performed less well, with a 63% and 50% score.

The Adobe Security Intelligence Team aims to garner feedback with the project and perhaps build a collection of pretrained analysis pipelines — a "model-zoo" — that can be distributed with future versions. The team comprises security engineers Vivek Malik and Kumar Vikramjeet, data scientist Tiberiu Boros, and technical lead Andrei Cotaie.

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
News
FluBot Malware's Rapid Spread May Soon Hit US Phones
Kelly Sheridan, Staff Editor, Dark Reading,  4/28/2021
Slideshows
7 Modern-Day Cybersecurity Realities
Steve Zurier, Contributing Writer,  4/30/2021
Commentary
How to Secure Employees' Home Wi-Fi Networks
Bert Kashyap, CEO and Co-Founder at SecureW2,  4/28/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2021-31755
PUBLISHED: 2021-05-07
An issue was discovered on Tenda AC11 devices with firmware through 02.03.01.104_CN. A stack buffer overflow vulnerability in /goform/setmac allows attackers to execute arbitrary code on the system via a crafted post request.
CVE-2021-31756
PUBLISHED: 2021-05-07
An issue was discovered on Tenda AC11 devices with firmware through 02.03.01.104_CN. A stack buffer overflow vulnerability in /gofrom/setwanType allows attackers to execute arbitrary code on the system via a crafted post request. This occurs when input vector controlled by malicious attack get copie...
CVE-2021-31757
PUBLISHED: 2021-05-07
An issue was discovered on Tenda AC11 devices with firmware through 02.03.01.104_CN. A stack buffer overflow vulnerability in /goform/setVLAN allows attackers to execute arbitrary code on the system via a crafted post request.
CVE-2021-31758
PUBLISHED: 2021-05-07
An issue was discovered on Tenda AC11 devices with firmware through 02.03.01.104_CN. A stack buffer overflow vulnerability in /goform/setportList allows attackers to execute arbitrary code on the system via a crafted post request.
CVE-2021-31458
PUBLISHED: 2021-05-07
This vulnerability allows remote attackers to execute arbitrary code on affected installations of Foxit Reader 10.1.1.37576. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file. The specific flaw exists within the handlin...