Application Security
12/19/2013
00:00 AM
Jeff Williams
Jeff Williams
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
100%
0%

Secure Code Starts With Measuring What Developers Know

I recently discovered I've been teaching blindly about application security. I assumed that I know what students need to learn. Nothing could be further from the truth.

Since 1999, I’ve taught over 2,000 developers, architects, and managers about application security. This is no small challenge, since the subject is almost totally ignored in most college curriculums and there is a lot to learn. In fact, the MITRE CWE Project lists over 1,000 different ­categories of security mistakes that developers can make. Many of these security quagmires are not immediately obvious and quite a few are downright diabolical. So I totally understand why developers don’t spend their off-hours researching the inner workings of "padding oracle" vulnerabilities and other security lore.

Still, we need developers to avoid making security mistakes that endanger their company and their users alike. Instructor-led training and e-learning are surprisingly effective and critical parts of an application security program. In one very large organization, we found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Despite many successes, I recently discovered that I’ve been teaching blindly. I have simply assumed what I thought my students needed to learn. We realized that measuring what students know, both before and after teaching, could help us provide more effective instruction. So we created “Secure Coder Analytics,” a measurement platform that analyzes a development team’s security knowledge. To ensure that developers don’t feel pressured, the tool protects participants’ anonymity.

Secure Coder Analytics draws questions from a pool of 500 questions that cover over 60 different secure coding subject areas. We have vetted the questions and answers for two years with real software development teams. Both the questions selected and the answers to those questions are fully randomized. While the questions are not easy, they have proven to be a reliable evaluation of a developer’s knowledge and skill in each area. Over 1,000 developers from around the world have participated, and the aggregate results are revealing.

  • The most important result is that only 59.5% of the questions are answered correctly. That’s a failing grade and helps explain the stunning prevalence of vulnerabilities in web applications.
  • The chart above shows the results for ten of the most critical security areas. While a few areas are passing, most are failing, and some are truly dismal.
  • Given that SQL Injection is the number one application security risk according to OWASP, it's surprising and encouraging to see that most developers have a firm understanding of what it takes to prevent it.

This chart above shows the five weakest and five strongest areas. The weakest are at or just above the "random guess" level, and are a real cause for concern. However, I am encouraged by the strong scores in important areas like "Preventing Forged Requests" and "Protecting Credentials."

The results for your organization will almost certainly be different. We’ve found that some organizations do quite well in areas that others totally fail. However, the results are fairly consistent within a particular organization. This suggests that different organizations are successfully teaching their developers about certain security areas. We hope to increase visibility and expand this training to cover what’s really important with Secure Coder Analytics, which I encourage you to try out for yourself. In the meantime, let’s chat about what you think your developers do and don’t know about application security in the comments.

 

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
M1ch43L
100%
0%
M1ch43L,
User Rank: Apprentice
12/22/2013 | 2:19:33 PM
SQL Injection
In my experience there are still a large number of developers who do not have a grasp on the SQL injection threat and proper coding techniques to prevent it. You'll hear "we use stored procedures so we're not vulnerable". Stored procedures just move the problem around. You'll also hear a variety of escape character strategies. In reality to properly prevent SQL injection vulnerabilities in your web applications you need to follow two important coding principles. First, never concatenate dynamic SQL from external input and second always use parameterized SQL anytime you must use external input in the application. 

However, even if you follow the above rules you're still potentially vulnerable to SQL injection. This is because 3rd party code running on your system could be vulnerable. Also, hackers can install malware to make your system vulnerable. Those who believe that simply fixing the applications will eliminate the SQL injection threat don't truly understand the threat.
Marilyn Cohodas
100%
0%
Marilyn Cohodas,
User Rank: Strategist
12/19/2013 | 5:01:45 PM
More bang for the buck
Very interesting article, Jeff. You make a strong case about where the emphasis in application security should begin and the numbers seem to bear you out:  

We found that projects where more than half the team members had received secure coding training, the number of vulnerabilities plummeted by 73 percent. That result is far superior to anything penetration testing programs or automated tools could hope to achieve.

Your point about each organization having it's own strengths and weaknesses is also somewhat of surprise.

So let me through this out to the communiity? What's holding you back from expanding your developer application security training? Let's talk more in the comments.  

Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Flash Poll
DevOps’ Impact on Application Security
DevOps’ Impact on Application Security
Managing the interdependency between software and infrastructure is a thorny challenge. Often, it’s a “developers are from Mars, systems engineers are from Venus” situation.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2012-1556
Published: 2014-09-12
Cross-site scripting (XSS) vulnerability in Synology Photo Station 5 for DiskStation Manager (DSM) 3.2-1955 allows remote attackers to inject arbitrary web script or HTML via the name parameter to photo/photo_one.php.

CVE-2014-2008
Published: 2014-09-12
SQL injection vulnerability in confirm.php in the mPAY24 payment module before 1.6 for PrestaShop allows remote attackers to execute arbitrary SQL commands via the TID parameter.

CVE-2014-2009
Published: 2014-09-12
The mPAY24 payment module before 1.6 for PrestaShop allows remote attackers to obtain credentials, the installation path, and other sensitive information via a direct request to api/curllog.log.

CVE-2014-4735
Published: 2014-09-12
Cross-site scripting (XSS) vulnerability in MyWebSQL 3.4 and earlier allows remote attackers to inject arbitrary web script or HTML via the table parameter to index.php.

CVE-2014-5259
Published: 2014-09-12
Cross-site scripting (XSS) vulnerability in cattranslate.php in the CatTranslate JQuery plugin in BlackCat CMS 1.0.3 and earlier allows remote attackers to inject arbitrary web script or HTML via the msg parameter.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
CISO Insider: An Interview with James Christiansen, Vice President, Information Risk Management, Office of the CISO, Accuvant