CIS, a coalition of enterprises, government agencies, universities, and vendors from around the world, in September provided an overview of the user-driven metrics project. The organization now has published a series of metrics for organizations to measure their key security operations -- incident management, vulnerability management, patch management, application security, configuration management, and financial metrics.
Metrics have begun gaining traction as a way for organizations to gauge their security risks, as well as how their existing tools and strategies are -- or are not -- working. A yardstick to measure just how secure an organization really is can help when it comes at budget time, experts say. Security consultancy Securosis, with initial financial backing from Microsoft, is in the process of creating a set of open-source metrics specifically for patch management. Securoris is gathering input in an open submission process for the so-called Project Quant metrics model, Version 1 of which is planned for release by the end of June.
Both CIS and Securosis were aware of one another's metrics projects, which could overlap on the patch management side.
"The purpose of this project is to define what a security incident is and how it can be counted and reported," says Bert Miuccio, CEO of CIS. "The challenge for information security professionals is to measure some aspect of security, like incident management, in an unambiguous way, and in a way that the measurement is repeatable over time so that different parts of the enterprise aren't measuring the same aspect of its security status in two different ways."
Among the around 20 different definitions/metrics by CIS is "Mean Time to Incident Recovery (MTIR)," or the ability of the organization to regain normal operations after a breach. MTIR is measured by the average amount of time elapsed between when an incident occurred and recovery.
Miuccio says CIS's vendor members -- which include Symantec, BMC Software, Belarc, CA, Configuresoft, nCircle, NetIQ, Opsware Network, Quest Software, Solidcore Systems, Tenable Network Security, and Tripwire -- are "in the process of examining" the metrics to determine, for instance, how to integrate them with their products' reporting features. "The most important aspect for the future adoption of the metrics is for the IT security software vendors to integrate these definitions into their tools and software solutions so they can automate the collection and reporting of these measures," he says.
But with budget pressures in a tight economy, it's unclear whether organizations will see adopting metrics as a potential cost-saving and efficiency strategy, or whether they just don't have the resources to adopt them. CIS's Miuccio says the coalition's metrics project is well-timed, "not only because of the condition of the economy, but also the state of security as a profession. There's an increasing emphasis on understanding the value that is derived from increasing security investments in the enterprise -- that was the rationale for staring this project in the first place."
Caroline Wong, chief of staff and manager of metrics for eBay, said in a statement that the metrics project "lays the groundwork for benchmarking, which has been impossible to date."
Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.