12:00 PM
Jeff Williams
Jeff Williams
Connect Directly
E-Mail vvv

Why Your Application Security Program May Backfire

You have to consider the human factor when you're designing security interventions, because the best intentions can have completely opposite consequences.

In security we have a saying: “Why do cars have brakes? So they can stop? No, so they can go fast!” Practiced badly, security can bring successful software projects to a screeching halt. Creating “security gates” for software projects, compliance reviews, and reporting phantom “false alarm” risks can kill a healthy relationship between security and development teams. But security doesn’t have to be about hindering business. Done right, application security programs are designed to get people working together in a way that is compatible with software development. The goal is to find solutions that allow business to go fast and be secure.

You know how a squirrel tries to get out the way of a car? Jump left. Zip right. Run directly under the tire. That’s how squirrels attempt to deal with a risk introduced by new technology (the car) using the predator avoidance mechanism designed by nature. It doesn’t work well because squirrels are marginally worse than humans at judging technology risk. Noted security expert Bruce Schneier argues that humans are fundamentally “bad at accurately assessing modern risk. We’re designed to exaggerate spectacular, strange, and rare events, and downplay ordinary, familiar, and common ones.”

But, if you really think about it, Mother Nature is just slow. We’re bad at IT risk because our defenses mostly evolved 100,000 years ago in small family groups in Africa. Given enough time, evolution adapts to new risks just fine. Species create remarkable defenses that are compatible with each organism. The question is: Can we speed up this process so that we can adapt more quickly to a rapidly changing technology risk environment?

Security and business evolve together
Usually, the software world just adopts new technology without thinking much about security. I’ve done hundreds of security reviews of products that were purchased without doing any security analysis -- and it’s usually not pretty. it’s not just products either. Organizations adopt new application frameworks, libraries, languages, and more without doing any security analysis. Even huge new features, like HTML5, show up in our browsers before any serious security work is done. This is a painful path forward, but eventually we hack and patch our way to achieve a “just barely good enough” level of security.

In my youth, I thought that enough design, architecture, and formal modeling could secure anything. But I’ve come to understand that the only way to security is through an evolutionary process with “builders” and “breakers” constantly challenging the status quo. Compliance efforts don’t engender this evolution, which is one reason they are often reviled. What’s not yet clear is exactly what does lead one organization to a great security culture, while another with the same practices struggles and makes little headway.

Safety and security make us take more risks
We put a lot of work into safety and security. But how do we know any “best-practices” are actually making us more secure? Before you answer, you should consider that a number of studies have shown that drivers of vehicles with ABS tend to drive faster, follow more closely, and brake later, accounting for the failure of ABS to result in any measurable improvement in road safety. So we are getting places faster without improving our safety.

This counter-intuitive outcome is called “Risk Compensation” or “Risk Homeostasis.” It turns out that people seem to be inherently wired with a certain level of risk tolerance.  Study after study supports this idea. Bike helmets have been show to make cyclists ride faster, even increasing the fatality rate in some studies. Seat belts make you drive less carefully. Ski helmets result in more aggressive skiing. Safer skydiving equipment, children’s toys, even condoms -- all show an almost intentional desire to bring the accident and fatality rates back to their original levels.

People’s perception is obviously critically important to changing behavior. What do you notice about the pedestrians in the picture below?

Did you notice that those people are walking around without any traffic signs, warning, road markings, curbs, and crosswalks? Security madness! If this were an audit, their area would be shut down and sent for remediation. However, this “shared space” was consciously designed to increase the level of uncertainty for drivers and other road users. Amazingly, this approach has been found to result in lower vehicle speeds and fewer road casualties.

The perception of protection
Wait… What!? Protection technology makes people take more risks? Removing safety markings makes roads safer? Yep. You have to consider the human factor when you’re designing security interventions. Your best intentions could have completely opposite and unintended consequences. For example, maybe your new web application firewall gives developers a false sense of security and they stop doing input validation. Or maybe you rely on automated security testing that has so many false alarms that people start to ignore the results.

These problems start when the perception of security isn’t in balance with the reality of security protections. When perception is greater than the protection, people have a false sense of security and take unnecessary risks. On the other hand, when the protection exceeds the perception, the business will shy away from profitable activities.

Creating a culture of accelerated security evolution with transparency
Ultimately, we all want to achieve that elusive culture that makes security a part of information technology without everybody ending up mad. I think we can all agree that building stuff and trying to make it secure later isn’t the right approach. But neither is blindly following a process model that just aggregates a bunch of guesses about what might work.

The path to security that works is rapid evolution – the “builders” and “breakers” working to push security forward. The key to speeding up this process is to make security transparent, so that the perception of security matches up with the reality. The keys to transparency are:

  • Starting with a model. You have to start with an “expected model” of what you think your defenses should be. It doesn’t have to be perfect, as your model will evolve over time.
  • Getting coverage. Verify your expected model across all your applications to an appropriate level of rigor. Establish a security sensor network that can monitor your entire portfolio.
  • Getting continuous. Security visibility has a terrifically short half-life. Today’s highly accelerated software processes demand real-time feedback to developers and other stakeholders.

If you have processes that aren’t demonstrably effective within this framework, cut them. You’ll get a more effective security program and probably save a lot of time and money. What are your techniques for creating a strong security culture? Let me know how you know it works in the comments! Good luck.

A pioneer in application security, Jeff Williams is the founder and CTO of Contrast Security, a revolutionary application security product that enhances software with the power to defend itself, check itself for vulnerabilities, and join a security command and control ... View Full Bio
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
7/16/2014 | 6:08:32 PM
Re: "shared space" philosophy versus reality
True.  But the point is that even though we aren't sure they help, we do all these security activities in the name of secure code.  I'm suggesting that we need to study the effectiveness of these activities because they *could* easily be undermining overall security.  It's not science to just "do what everyone else is doing" -- that ends up with expensive programs that everyone hates (sound familiar?)
User Rank: Author
7/16/2014 | 6:05:42 PM
Re: Why Your Application Security Program May Backfire
Absolutely agree that taking a positive approach to security is easier, faster, and more accurate.  It's the key to getting continuous.
User Rank: Author
7/11/2014 | 9:51:33 AM
Re: Why Your Application Security Program May Backfire
Nice read @JeffWilliams.  I tend to agree that the "know-do" gap of what users know versus what they do largely contributes of what you're discussing.  This begs the questions do we need to improve our security awareness programs in an effort to reduce this gap?

Well if you look at bike helmets or vehicle ABS in the context of managing security risk, perhaps this stems from how we continue to use traditional security approaches in a very different threat landscape.

Going to your comment about "Removing safety markings makes roads safer" brings me back to the idea that the use of positive security is much more effective in reducing risks in today's technology world.
User Rank: Apprentice
7/6/2014 | 12:07:30 PM
"shared space" philosophy versus reality
There is growing evidence that initial claims for reduced traffic sppeds and greater pedestrian safety in "shared spaces" is overstated.

See: Shared space - research, policy and problems - http://eprints.uwe.ac.uk/17937/8/tran1200047h.pdf

There is evidence that pedestrians and bicyclists "feel" less safe and adopt avoidance behaviors, sometimes avoiding "shared spaces" altogether.  Many advocates for the blind and physically impaired raise strong voices against "shared spaces" as extremely dangerous places to navigate, again urging those on behalf of whom they advocate to avoid such spaces. 

Seems that "sharing" favors drivers more than pedestrians and other users.
Valentine's Emails Laced with Gandcrab Ransomware
Kelly Sheridan, Staff Editor, Dark Reading,  2/14/2019
High Stress Levels Impacting CISOs Physically, Mentally
Jai Vijayan, Freelance writer,  2/14/2019
Mozilla, Internet Society and Others Pressure Retailers to Demand Secure IoT Products
Curtis Franklin Jr., Senior Editor at Dark Reading,  2/14/2019
Register for Dark Reading Newsletters
White Papers
Current Issue
5 Emerging Cyber Threats to Watch for in 2019
Online attackers are constantly developing new, innovative ways to break into the enterprise. This Dark Reading Tech Digest gives an in-depth look at five emerging attack trends and exploits your security team should look out for, along with helpful recommendations on how you can prevent your organization from falling victim.
Flash Poll
How Enterprises Are Attacking the Cybersecurity Problem
How Enterprises Are Attacking the Cybersecurity Problem
Data breach fears and the need to comply with regulations such as GDPR are two major drivers increased spending on security products and technologies. But other factors are contributing to the trend as well. Find out more about how enterprises are attacking the cybersecurity problem by reading our report today.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-02-19
In DedeCMS 5.7SP2, attackers can upload a .php file to the uploads/ directory (without being blocked by the Web Application Firewall), and then execute this file, via this sequence of steps: visiting the management page, clicking on the template, clicking on Default Template Management, clicking on ...
PUBLISHED: 2019-02-18
Stack-based buffer overflow in the strip_vt102_codes function in TinTin++ 2.01.6 and WinTin++ 2.01.6 allows remote attackers to execute arbitrary code by sending a long message to the client.
PUBLISHED: 2019-02-18
The seadroid (aka Seafile Android Client) application through 2.2.13 for Android always uses the same Initialization Vector (IV) with Cipher Block Chaining (CBC) Mode to encrypt private data, making it easier to conduct chosen-plaintext attacks or dictionary attacks.
PUBLISHED: 2019-02-18
SolarWinds Orion NPM before 12.4 suffers from a SYSTEM remote code execution vulnerability in the OrionModuleEngine service. This service establishes a NetTcpBinding endpoint that allows remote, unauthenticated clients to connect and call publicly exposed methods. The InvokeActionMethod method may b...
PUBLISHED: 2019-02-18
An issue was discovered in WTCMS 1.0. It allows remote attackers to execute arbitrary PHP code by going to the "Setting -> Mailbox configuration -> Registration email template" screen, and uploading an image file, as demonstrated by a .php filename and the "Content-Type: image/g...