Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Bringing Science to the Debate

It's time to get an account of whether proof-of-concept/exploit code actually helps or hurts users

Repeat after me: "We don't know."

We don't know if responsible disclosure is better or worse than full disclosure. We don't know if releasing proof of concept or exploit code helps us or the bad guys, more or less. We don't know how the silent majority of IT workers want us in the security community to handle these issues, nor if their opinion is self-destructive or imminently practical. When you get down to it, we don't really know a fracking thing.

Sure, we all have plenty of anecdotal evidence to support our personal positions. We can all cite cases of this or that vendor tirelessly defending its customers, or putting them at mortal risk based on their handling of some vulnerability. We all know someone that suffered real losses at the hands of the latest random Metasploit exploit module, and someone else who used it to close critical holes in their security defenses before the bad guys made it in. We all talk about Blaster, Code Red, and other past incidents like they have any relevance in today's world, which we all also admit has changed completely from a few years ago.

There’s a word for picking and choosing examples to support a pre-existing belief without any scientific basis. It's called religion.

If Dan Kaminsky's recent unorthodox disclosure of the DNS vulnerability has done nothing else, it's polarized the IT community into a dozen discrete corners of our never-ending disclosure debate. Rain Forrest Puppy may have first formalized responsible disclosure back in the late 1990s, but we're far from any industry consensus. As an analyst, my customers were on all sides -- users, vendors, and researchers -- and I rarely saw agreement in any single demographic, never mind between them.

In a previous column here at Dark Reading I called the disclosure debate dead and received a bit of flack over that line, but the truth is the debate just hadn't advanced in meaningful way. It was nothing more than the same tired arguments on all sides.

I propose that it's long past time we brought some current science into the game. It's time to move past anecdotal evidence or one-off cases into wider-ranging realm of epidemiological studies. It's time to ask the users what they want, while developing risk metrics to allow them to make informed decisions despite their personal opinions. We may not reach definitive conclusions, and even if we do, they probably won't last nor change the minds of the truly religious. But it's always better to seek more data than to dismiss it before we even see it.

To that end, I'd like to take a baby step, with a small poll we're hosting here on Dark Reading. I've seen a bunch of polls on various blogs over this DNS issue, but no one has asked people to respond based on their role in the industry: end user, researcher, or vendor. Rather than a broad disclosure opinion poll, we're focusing on a single issue inspired by recent events -- the release of public exploit/proof of concept code at the same time vulnerability is disclosed. While we know this kind of a poll isn't statistically valid, it's at least a start.

Please take a moment to participate in our quick poll here at this link.

I was personally critical of HD Moore for releasing Metasploit exploit code for the DNS vulnerability so quickly after patches were available. My personal opinion is that Metasploit is so widely used, and so easy to use, that it empowers attackers on a scale far beyond the inevitable one-off exploits being rapidly developed across the globe. The scales are different for Metasploit, and while we need those modules eventually for testing, releasing weaponized versions too quickly hurts us more than it helps us.

But that's my opinion. In a recent blog post Richard Bejtlich made a cogent argument for the release of these same modules. It helped him, hurt some other end user clients I've talked with, but none of us really knows what's best on a broad scale.

Andrew Jaquith, an analyst at The Yankee Group, agrees. "The debates about full versus responsible disclosure, proof-of-concept code, and attack/exploit frameworks are passionate. People argue their points of view with incredible conviction -- but without any empirical evidence one way or the other,” Jaquith says. “What we need are metrics that show the effect -- or not -- of PoC/exploit code on customers. Is it helping them detect problems and fix them? Or does it increase their exposure to attack? The debate needs move from philosophizing to facts, and from dogma to data."

A group of us, including Andrew, have decided to take action and start collecting data. This poll is just a small way you can contribute, and if you are interested in being more involved you can email me at [email protected]. We don't know exactly what we're doing, nor how, but we do know it's time to get organized and take action.

It's time for more science, and less religion.

— Rich Mogull is founder of Securosis LLC and a former security industry analyst for Gartner Inc. Special to Dark Reading.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Why Cyber-Risk Is a C-Suite Issue
Marc Wilczek, Digital Strategist & CIO Advisor,  11/12/2019
Unreasonable Security Best Practices vs. Good Risk Management
Jack Freund, Director, Risk Science at RiskLens,  11/13/2019
Breaches Are Inevitable, So Embrace the Chaos
Ariel Zeitlin, Chief Technology Officer & Co-Founder, Guardicore,  11/13/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
Navigating the Deluge of Security Data
In this Tech Digest, Dark Reading shares the experiences of some top security practitioners as they navigate volumes of security data. We examine some examples of how enterprises can cull this data to find the clues they need.
Flash Poll
Rethinking Enterprise Data Defense
Rethinking Enterprise Data Defense
Frustrated with recurring intrusions and breaches, cybersecurity professionals are questioning some of the industrys conventional wisdom. Heres a look at what theyre thinking about.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-11-15
An issue was discovered in Marvell 88W8688 Wi-Fi firmware before version p52, as used on Tesla Model S/X vehicles manufactured before March 2018, via the Parrot Faurecia Automotive FC6050W module. A heap-based buffer overflow allows remote attackers to cause a denial of service or execute arbitrary ...
PUBLISHED: 2019-11-15
An issue was discovered in Marvell 88W8688 Wi-Fi firmware before version p52, as used on Tesla Model S/X vehicles manufactured before March 2018, via the Parrot Faurecia Automotive FC6050W module. A stack overflow could lead to denial of service or arbitrary code execution.
PUBLISHED: 2019-11-15
On version 14.0.0-, BIG-IP virtual servers with TLSv1.3 enabled may experience a denial of service due to undisclosed incoming messages.
PUBLISHED: 2019-11-15
On BIG-IP 14.1.0-14.1.2, 14.0.0-14.0.1, and 13.1.0-13.1.1, undisclosed HTTP requests may consume excessive amounts of systems resources which may lead to a denial of service.
PUBLISHED: 2019-11-15
When the BIG-IP APM 14.1.0-14.1.2, 14.0.0-14.0.1, 13.1.0-, 12.1.0-, or 11.5.1-11.6.5 system processes certain requests, the APD/APMD daemon may consume excessive resources.