Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


01:00 PM
Nick Heudecker
Nick Heudecker
Connect Directly
E-Mail vvv

An Observability Pipeline Could Save Your SecOps Team

Traditional monitoring approaches are proving brittle as security operations teams need better visibility into dynamic environments.

Security analysts are struggling with two opposing challenges: too much data and too little of the right data. According to a recent survey from the Ponemon Institute, 71% of respondents cite information overload as a key stress factor in their work; 63% also call out a lack of visibility into the network and infrastructure as a stressor.

Related Content:

Mainframe Security Automation Is Not a Luxury

Special Report: Understanding Your Cyber Attackers

Comparing Different AI Approaches to Email Security

Conventional concerns, like the growing complexity of distributed denial-of-service attacks and negligent insiders, complicate today's security operations center (SOC) environment. In addition, cloud-native applications deployed on containers and other transient infrastructure are another factor in these challenges. Applications, and the infrastructure they run on, are more dynamic and ephemeral than before, and that comes with a level of complexity that traditional monitoring hasn't grappled with.

Shifting to Observable Systems
Over the last 18 months, operations teams, including security operations personnel, are talking about the shift from static monitoring to dynamic observability. While monitoring focuses on the health of individual components, observability provides fine-grained visibility into why systems behave the way they do. Observability is the characteristic of software, infrastructure, and systems allowing questions about their behavior to be asked and answered. Contrast this with monitoring, which forces predefined questions about systems into a set of blinking dashboards that may or may not tell you what's going on in your environment.

However, observability isn't a thing you can buy. No single tool provides all the benefits of observable systems. Teams must build observable systems, starting with embedding the concept into applications and infrastructure in the form of logs, metrics, and traces. Combining that data with change logs, IT service management data, and network traffic gives teams the big picture but also enables drilling down into details. Some early implementations of observability also include social media feeds to uncover customer problems with applications before those signals propagate into metrics-based dashboards.

Changing Cultures
Complexity is only one facet pushing observability. Beyond the shortcomings in traditional monitoring, observability is becoming important as security operations teams work cross-functionally. Today's SOC teams interact with infrastructure and operations as well as DevOps groups, each with their own tooling and analytics platforms. This interaction style is something security operations teams may not have done in the past. It introduces friction between these teams around what various data sets mean or what a correct outcome even looks like. Observability helps resolve these tribal views of data by delivering the right data to respective platforms.

Implementing Observability Infrastructure
With instrumented systems, delivering data to the right platforms becomes a challenge. The concept of an observability pipeline, popularized by Tyler Treat, is becoming a critical element in implementing observability because it decouples the sources, like applications and infrastructure, from destinations, like log analytics and SIEM platforms. Most organizations have 10 or more tools for security and analytics, and nearly half believe they need more. Abstracting data analysis and use from how it's collected gives teams flexibility in delivering data. It also allows for fine-grained optimization of the data sources through redaction, filtering, and reducing data volumes.

The last component, after observable instrumentation and the pipeline, is exploring data. Coming from the data and analytics space, I equate traditional monitoring to data warehousing. In both data warehousing and monitoring, you know what data you're ingesting and the reports or dashboards you're creating. You have a collection of known questions over known data. It may be expensive and inflexible, but it's also reliable and well understood.

Observability is more like a data lake. With a data lake, you don't know what questions you'll ask, but you fill the lake with data and organize it to prepare for future questions. If a data warehouse is for known questions over known data, a data lake is for unknown questions over unknown data. It's often helpful to think of a data lake as a question development environment since you're creating the questions you want to ask as you're exploring the data. Unlike a conventional data lake supporting data scientists optimizing for SQL and Python, an observability data lake optimizes for search.

Security analysts have too much data to manage and analyze, and they still don't have all the data they need to get visibility into their environment. Traditional approaches such as monitoring may have solved some problems in the past, but they are quickly being outpaced by changes in the evolving IT ecosystem led by shifts like cloud-native or container-based infrastructure. Teams need a fresh approach when tackling complexity. This is where observable systems come into play. By building modern systems with observability in mind, you can better future proof your systems when questions arise and evolve over time. An observability pipeline is a critical piece of the puzzle. It gives you the flexibility to capture the universe of data you need and deliver it cleaned and formatted to the myriad of tools your teams need.

This column was written with Bryan Turriff, Cribl's Director of Product Marketing.

Nick Heudecker is the Senior Director of Market Strategy at Cribl. Prior to joining Cribl, he spent over seven years as an industry analyst at Gartner, covering the data and analytics market. With over 20 years of experience, he has led engineering and product teams across ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
How SolarWinds Busted Up Our Assumptions About Code Signing
Dr. Jethro Beekman, Technical Director,  3/3/2021
'ObliqueRAT' Now Hides Behind Images on Compromised Websites
Jai Vijayan, Contributing Writer,  3/2/2021
Attackers Turn Struggling Software Projects Into Trojan Horses
Robert Lemos, Contributing Writer,  2/26/2021
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-03-05
In SPIRE before versions 0.8.5, 0.9.4, 0.10.2, 0.11.3 and 0.12.1, the "aws_iid" Node Attestor improperly normalizes the path provided through the agent ID templating feature, which may allow the issuance of an arbitrary SPIFFE ID within the same trust domain, if the attacker controls the v...
PUBLISHED: 2021-03-05
An issue was discovered in the Linux kernel through 5.11.3, as used with Xen PV. A certain part of the netback driver lacks necessary treatment of errors such as failed memory allocations (as a result of changes to the handling of grant mapping errors). A host OS denial of service may occur during m...
PUBLISHED: 2021-03-05
An issue was discovered in the Linux kernel 5.9.x through 5.11.3, as used with Xen. In some less-common configurations, an x86 PV guest OS user can crash a Dom0 or driver domain via a large amount of I/O activity. The issue relates to misuse of guest physical addresses when a configuration has CONFI...
PUBLISHED: 2021-03-05
An issue was discovered in OSSEC 3.6.0. An uncontrolled recursion vulnerability in os_xml.c occurs when a large number of opening and closing XML tags is used. Because recursion is used in _ReadElem without restriction, an attacker can trigger a segmentation fault once unmapped memory is reached.
PUBLISHED: 2021-03-05
This affects the package xmlhttprequest before 1.7.0; all versions of package xmlhttprequest-ssl. Provided requests are sent synchronously (async=False on xhr.open), malicious user input flowing into xhr.send could result in arbitrary code being injected and run.