What's the Difference Between 'Observability' and 'Visibility' in Security?

Question: What's the difference between "observability" and "visibility" in security?  

Joe Vadakkan, global cloud security leader, Optiv Security: As enterprises digitally transform, they are naturally undergoing security modernization as well. These efforts are dependent on mapping various security elements to keep up with dynamic environments in cloud, K8 clusters, infrastructure-as-code (IaC) deployment, and third-party toolsets. To drive holistic security success, though, we have to start with the interlinking of visibility and observability.

"Visibility" is achieved through monitoring systems, networks, applications, performance, through-point, or several-point solutions and aggregating that data. In the past, organizations wanted visibility into everything and went on shopping sprees for every point solution product out there. API-driven architecture allowed us to aggregate more logs, which gave us a single pane of glass and the first generation of security analytics. It also turned aggregated security logs into a data landfill.

"Observability" expands on that monitoring and enables correlation and inspection of the raw data to provide much deeper insights. With the proper instrumentation, observability allows an enterprise, both inside and outside of the security organization, to solve an extensive number of use cases. Observability requires several elements of logs, metrics, and deep tracing. All data from security, business, and technology sources is pipelined for enrichment and modeling. It opens us up to the second generation of analytics. We’re now able to mine the data, build patterns, make useful calculations out of artificial intelligence and machine learning samples, and improve remediation with proactive and reactive hyper-automation.

In my opinion, observability is the latest, most important fabric within a security modernization program. The more we expand the baseline understanding of our systems, the more proactive we can be in continuously improving our efforts.