Application Security
12/19/2013
00:00 AM
Jeff Williams
Jeff Williams
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
100%
0%

Secure Code Starts With Measuring What Developers Know

I recently discovered I've been teaching blindly about application security. I assumed that I know what students need to learn. Nothing could be further from the truth.

Since 1999, I’ve taught over 2,000 developers, architects, and managers about application security. This is no small challenge, since the subject is almost totally ignored in most college curriculums and there is a lot to learn. In fact, the MITRE CWE Project lists over 1,000 different ­categories of security mistakes that developers can make. Many of these security quagmires are not immediately obvious and quite a few are downright diabolical. So I totally understand why developers don’t spend their off-hours researching the inner workings of "padding oracle" vulnerabilities and other security lore.

Still, we need developers to avoid making security mistakes that endanger their company and their users alike. Instructor-led training and e-learning are surprisingly effective and critical parts of an application security program. In one very large organization, we found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Despite many successes, I recently discovered that I’ve been teaching blindly. I have simply assumed what I thought my students needed to learn. We realized that measuring what students know, both before and after teaching, could help us provide more effective instruction. So we created “Secure Coder Analytics,” a measurement platform that analyzes a development team’s security knowledge. To ensure that developers don’t feel pressured, the tool protects participants’ anonymity.

Secure Coder Analytics draws questions from a pool of 500 questions that cover over 60 different secure coding subject areas. We have vetted the questions and answers for two years with real software development teams. Both the questions selected and the answers to those questions are fully randomized. While the questions are not easy, they have proven to be a reliable evaluation of a developer’s knowledge and skill in each area. Over 1,000 developers from around the world have participated, and the aggregate results are revealing.

  • The most important result is that only 59.5% of the questions are answered correctly. That’s a failing grade and helps explain the stunning prevalence of vulnerabilities in web applications.
  • The chart above shows the results for ten of the most critical security areas. While a few areas are passing, most are failing, and some are truly dismal.
  • Given that SQL Injection is the number one application security risk according to OWASP, it's surprising and encouraging to see that most developers have a firm understanding of what it takes to prevent it.

This chart above shows the five weakest and five strongest areas. The weakest are at or just above the "random guess" level, and are a real cause for concern. However, I am encouraged by the strong scores in important areas like "Preventing Forged Requests" and "Protecting Credentials."

The results for your organization will almost certainly be different. We’ve found that some organizations do quite well in areas that others totally fail. However, the results are fairly consistent within a particular organization. This suggests that different organizations are successfully teaching their developers about certain security areas. We hope to increase visibility and expand this training to cover what’s really important with Secure Coder Analytics, which I encourage you to try out for yourself. In the meantime, let’s chat about what you think your developers do and don’t know about application security in the comments.

 

Comment  | 
Print  | 
More Insights
Comments
Oldest First  |  Newest First  |  Threaded View
Marilyn Cohodas
100%
0%
Marilyn Cohodas,
User Rank: Strategist
12/19/2013 | 5:01:45 PM
More bang for the buck
Very interesting article, Jeff. You make a strong case about where the emphasis in application security should begin and the numbers seem to bear you out:  

We found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Your point about each organization having it's own strengths and weaknesses is also somewhat of surprise.

So let me through this out to the communiity? What's holding you back from expanding your developer application security training? Let's talk more in the comments.  

M1ch43L
100%
0%
M1ch43L,
User Rank: Apprentice
12/22/2013 | 2:19:33 PM
SQL Injection
In my experience there are still a large number of developers who do not have a grasp on the SQL injection threat and proper coding techniques to prevent it. You'll hear "we use stored procedures so we're not vulnerable". Stored procedures just move the problem around. You'll also hear a variety of escape character strategies. In reality to properly prevent SQL injection vulnerabilities in your web applications you need to follow two important coding principles. First, never concatenate dynamic SQL from external input and second always use parameterized SQL anytime you must use external input in the application. 

However, even if you follow the above rules you're still potentially vulnerable to SQL injection. This is because 3rd party code running on your system could be vulnerable. Also, hackers can install malware to make your system vulnerable. Those who believe that simply fixing the applications will eliminate the SQL injection threat don't truly understand the threat.
Register for Dark Reading Newsletters
White Papers
Flash Poll
Current Issue
Cartoon
DevOps’ Impact on Application Security
DevOps’ Impact on Application Security
Managing the interdependency between software and infrastructure is a thorny challenge. Often, it’s a “developers are from Mars, systems engineers are from Venus” situation.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2014-4725
Published: 2014-07-27
The MailPoet Newsletters (wysija-newsletters) plugin before 2.6.7 for WordPress allows remote attackers to bypass authentication and execute arbitrary PHP code by uploading a crafted theme using wp-admin/admin-post.php and accessing the theme in wp-content/uploads/wysija/themes/mailp/.

CVE-2014-4726
Published: 2014-07-27
Unspecified vulnerability in the MailPoet Newsletters (wysija-newsletters) plugin before 2.6.8 for WordPress has unspecified impact and attack vectors.

CVE-2014-2363
Published: 2014-07-26
Morpho Itemiser 3 8.17 has hardcoded administrative credentials, which makes it easier for remote attackers to obtain access via a login request.

CVE-2014-2625
Published: 2014-07-26
Directory traversal vulnerability in the storedNtxFile function in HP Network Virtualization 8.6 (aka Shunra Network Virtualization) allows remote attackers to read arbitrary files via crafted input, aka ZDI-CAN-2023.

CVE-2014-2626
Published: 2014-07-26
Directory traversal vulnerability in the toServerObject function in HP Network Virtualization 8.6 (aka Shunra Network Virtualization) allows remote attackers to create files, and consequently execute arbitrary code, via crafted input, aka ZDI-CAN-2024.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Sara Peters hosts a conversation on Botnets and those who fight them.