Application Security
12/19/2013
00:00 AM
Jeff Williams
Jeff Williams
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
100%
0%

Secure Code Starts With Measuring What Developers Know

I recently discovered I've been teaching blindly about application security. I assumed that I know what students need to learn. Nothing could be further from the truth.

Since 1999, I’ve taught over 2,000 developers, architects, and managers about application security. This is no small challenge, since the subject is almost totally ignored in most college curriculums and there is a lot to learn. In fact, the MITRE CWE Project lists over 1,000 different ­categories of security mistakes that developers can make. Many of these security quagmires are not immediately obvious and quite a few are downright diabolical. So I totally understand why developers don’t spend their off-hours researching the inner workings of "padding oracle" vulnerabilities and other security lore.

Still, we need developers to avoid making security mistakes that endanger their company and their users alike. Instructor-led training and e-learning are surprisingly effective and critical parts of an application security program. In one very large organization, we found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Despite many successes, I recently discovered that I’ve been teaching blindly. I have simply assumed what I thought my students needed to learn. We realized that measuring what students know, both before and after teaching, could help us provide more effective instruction. So we created “Secure Coder Analytics,” a measurement platform that analyzes a development team’s security knowledge. To ensure that developers don’t feel pressured, the tool protects participants’ anonymity.

Secure Coder Analytics draws questions from a pool of 500 questions that cover over 60 different secure coding subject areas. We have vetted the questions and answers for two years with real software development teams. Both the questions selected and the answers to those questions are fully randomized. While the questions are not easy, they have proven to be a reliable evaluation of a developer’s knowledge and skill in each area. Over 1,000 developers from around the world have participated, and the aggregate results are revealing.

  • The most important result is that only 59.5% of the questions are answered correctly. That’s a failing grade and helps explain the stunning prevalence of vulnerabilities in web applications.
  • The chart above shows the results for ten of the most critical security areas. While a few areas are passing, most are failing, and some are truly dismal.
  • Given that SQL Injection is the number one application security risk according to OWASP, it's surprising and encouraging to see that most developers have a firm understanding of what it takes to prevent it.

This chart above shows the five weakest and five strongest areas. The weakest are at or just above the "random guess" level, and are a real cause for concern. However, I am encouraged by the strong scores in important areas like "Preventing Forged Requests" and "Protecting Credentials."

The results for your organization will almost certainly be different. We’ve found that some organizations do quite well in areas that others totally fail. However, the results are fairly consistent within a particular organization. This suggests that different organizations are successfully teaching their developers about certain security areas. We hope to increase visibility and expand this training to cover what’s really important with Secure Coder Analytics, which I encourage you to try out for yourself. In the meantime, let’s chat about what you think your developers do and don’t know about application security in the comments.

 

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
M1ch43L
100%
0%
M1ch43L,
User Rank: Apprentice
12/22/2013 | 2:19:33 PM
SQL Injection
In my experience there are still a large number of developers who do not have a grasp on the SQL injection threat and proper coding techniques to prevent it. You'll hear "we use stored procedures so we're not vulnerable". Stored procedures just move the problem around. You'll also hear a variety of escape character strategies. In reality to properly prevent SQL injection vulnerabilities in your web applications you need to follow two important coding principles. First, never concatenate dynamic SQL from external input and second always use parameterized SQL anytime you must use external input in the application. 

However, even if you follow the above rules you're still potentially vulnerable to SQL injection. This is because 3rd party code running on your system could be vulnerable. Also, hackers can install malware to make your system vulnerable. Those who believe that simply fixing the applications will eliminate the SQL injection threat don't truly understand the threat.
Marilyn Cohodas
100%
0%
Marilyn Cohodas,
User Rank: Strategist
12/19/2013 | 5:01:45 PM
More bang for the buck
Very interesting article, Jeff. You make a strong case about where the emphasis in application security should begin and the numbers seem to bear you out:  

We found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Your point about each organization having it's own strengths and weaknesses is also somewhat of surprise.

So let me through this out to the communiity? What's holding you back from expanding your developer application security training? Let's talk more in the comments.  

Register for Dark Reading Newsletters
White Papers
Flash Poll
Current Issue
Cartoon
DevOpsí Impact on Application Security
DevOpsí Impact on Application Security
Managing the interdependency between software and infrastructure is a thorny challenge. Often, itís a ďdevelopers are from Mars, systems engineers are from VenusĒ situation.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-6306
Published: 2014-08-22
Unspecified vulnerability on IBM Power 7 Systems 740 before 740.70 01Ax740_121, 760 before 760.40 Ax760_078, and 770 before 770.30 01Ax770_062 allows local users to gain Service Processor privileges via unknown vectors.

CVE-2014-0232
Published: 2014-08-22
Multiple cross-site scripting (XSS) vulnerabilities in framework/common/webcommon/includes/messages.ftl in Apache OFBiz 11.04.01 before 11.04.05 and 12.04.01 before 12.04.04 allow remote attackers to inject arbitrary web script or HTML via unspecified vectors, which are not properly handled in a (1)...

CVE-2014-3525
Published: 2014-08-22
Unspecified vulnerability in Apache Traffic Server 4.2.1.1 and 5.x before 5.0.1 has unknown impact and attack vectors, possibly related to health checks.

CVE-2014-3563
Published: 2014-08-22
Multiple unspecified vulnerabilities in Salt (aka SaltStack) before 2014.1.10 allow local users to have an unspecified impact via vectors related to temporary file creation in (1) seed.py, (2) salt-ssh, or (3) salt-cloud.

CVE-2014-3587
Published: 2014-08-22
Integer overflow in the cdf_read_property_info function in cdf.c in file through 5.19, as used in the Fileinfo component in PHP before 5.4.32 and 5.5.x before 5.5.16, allows remote attackers to cause a denial of service (application crash) via a crafted CDF file. NOTE: this vulnerability exists bec...

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Three interviews on critical embedded systems and security, recorded at Black Hat 2014 in Las Vegas.