Certifiable

Where do good security people come from? In my book Software Security, I hype up the idea that they come from all sorts of widely divergent backgrounds:

So that's good, right? But it also leaves plenty of room for hacks and hucksters. In fact, I think it is clear that there remains far too much room in our field for snake oil, scare tactics, and plain old bull feathers. There may be some light at the end of the long dark tunnel now that a number of professional certifications are available, but not all certifications are good.

CISSP: Shysters or Knowledgeable Practitioners?
Because the field is young, it can be difficult for inhabitants of Wild West outposts to distinguish purveyors of snake oil from experienced brain surgeons. To combat this difficulty, the idea of certifying practitioners has been adopted. The by-now widespread CISSP certification is one great example. A CISSP certification is an indicator that someone has mastered a common body of practical security knowledge and at least has a pulse. Tests are used to separate those who deserve a CISSP (that is, those who passed the test) from the unwashed masses. Education and certification companies make money both selling test preparation training and administering the test.

To harken back to our doctor analogy for a moment, the problem with the CISSP scheme is that certification doesn’t do much to address the problem of discriminating self-taught homeopaths from doctors or surgeons. In fact, most security surgeons I know scoff at the CISSP certification as not at all useful and a waste of their very precious time. These specialists instead rely on well established academic credentials such as computer science degrees of various levels with a specialty in security from a well-respected academic institution (think Princeton or Purdue).

Because academia can't produce enough surgeons to satisfy all security demands (and indeed because entire armies of less specialized "healthcare professionals" are necessary), the idea of a certification makes plenty of practical sense.

This creates a multi-tiered system of the same sort usually found among professionals, academics, and scientists, where workaday practitioners differ from specialists and have different credentials. In real life, sometimes you need a specialist, and sometimes you don't.

Some kind of balance seems to have been struck in network security, with academics and practitioners both seeking and finding the kinds of credentials that they need. As the field grows up, more standard academic credentials are very likely to replace certifications such as the CISSP (at least for some classes of workers). But because network security is mostly about network operations, things seem to be working just fine.

What's on Your Belly?
When it comes to software security, things are in much worse shape. The problem is that there is no credentialing system for software engineers. In fact, almost anyone can write software today (and many do, including my 12 year old son). Academics are doing what they can to create a class of software professionals, but the going is slow and much work remains to be done. Software security is only the most remote afterthought. What we need is some kind of academic credentialing system for software engineers (which will hopefully include decent coverage of software security touchpoints such as code review and architectural risk analysis).

Into this very obvious vacuum comes a proposed credentialing system for software developers envisioned by SANS (who created a GIAC certification system that competes with the CISSP system but that in many ways is much more detailed). The GSSP (GIAC Secure Software Programmer) is a certification system that purports to tell a developer who can write secure software from one who can't.

I am reminded of one Sylvester McMonkey McBean, the key protagonist of Dr. Seuss's Sneetches story. Just to remind you, here is a snippet from that story:

In the story, McBean devised a system for selling stars to the poor discriminated-against Plain-Belly Sneetches. Many such Sneetches rushed to the opportunity to become credentialed. Of course the original Star-Bellies were incensed and the very clever McBean had an answer for that: star removal. Chaos ensued, Sneetches cycled, and McBean got rich.

The analogy between Sneetches and developers with security stars breaks down pretty quickly, but what makes it salient for me is McBean. As far as I can tell from a careful study of the guideline documents produced by SANS, the only thing you can tell about a developer with a GSSP is that they once had $400 to pay for the test.

The multiple choice test itself is one of the problems. I have discussed the idea of using multiple choice to discriminate knowledgeable developers from clueless developers (like the SANS test does) with many professors of computer science. Not one of them thought it was possible.

Some hold out hope that making developers think at all about security will be one of the side effects of the certification idea. I hope they are right, but I remain skeptical. Of course anything that serves to raise awareness of the software security issue is probably good for us, so let the fun and games begin!

Ultimately, the real problem is that in order to be a good software security developer, you need to be a good developer first. There are no tests for that. I wish there were.

Credentials and Other Metrics
Hopefully, this notion of measuring people and capability will be one of the topics of discussion at this year's MetriCon conference. Even if the people question does not come up, I am sure that many other interesting security metrics questions will. I plan to write more on that in a future column, but while you are waiting, please rush out and buy a copy of Andrew Jaquith's great book, Security Metrics: Replacing Fear Uncertainty and Doubt.

— Gary McGraw is CTO of Cigital Inc. Special to Dark Reading