Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


12:00 PM
Jeff Williams
Jeff Williams
Connect Directly
E-Mail vvv

Why Your Application Security Program May Backfire

You have to consider the human factor when you're designing security interventions, because the best intentions can have completely opposite consequences.

In security we have a saying: “Why do cars have brakes? So they can stop? No, so they can go fast!” Practiced badly, security can bring successful software projects to a screeching halt. Creating “security gates” for software projects, compliance reviews, and reporting phantom “false alarm” risks can kill a healthy relationship between security and development teams. But security doesn’t have to be about hindering business. Done right, application security programs are designed to get people working together in a way that is compatible with software development. The goal is to find solutions that allow business to go fast and be secure.

You know how a squirrel tries to get out the way of a car? Jump left. Zip right. Run directly under the tire. That’s how squirrels attempt to deal with a risk introduced by new technology (the car) using the predator avoidance mechanism designed by nature. It doesn’t work well because squirrels are marginally worse than humans at judging technology risk. Noted security expert Bruce Schneier argues that humans are fundamentally “bad at accurately assessing modern risk. We’re designed to exaggerate spectacular, strange, and rare events, and downplay ordinary, familiar, and common ones.”

But, if you really think about it, Mother Nature is just slow. We’re bad at IT risk because our defenses mostly evolved 100,000 years ago in small family groups in Africa. Given enough time, evolution adapts to new risks just fine. Species create remarkable defenses that are compatible with each organism. The question is: Can we speed up this process so that we can adapt more quickly to a rapidly changing technology risk environment?

Security and business evolve together
Usually, the software world just adopts new technology without thinking much about security. I’ve done hundreds of security reviews of products that were purchased without doing any security analysis -- and it’s usually not pretty. it’s not just products either. Organizations adopt new application frameworks, libraries, languages, and more without doing any security analysis. Even huge new features, like HTML5, show up in our browsers before any serious security work is done. This is a painful path forward, but eventually we hack and patch our way to achieve a “just barely good enough” level of security.

In my youth, I thought that enough design, architecture, and formal modeling could secure anything. But I’ve come to understand that the only way to security is through an evolutionary process with “builders” and “breakers” constantly challenging the status quo. Compliance efforts don’t engender this evolution, which is one reason they are often reviled. What’s not yet clear is exactly what does lead one organization to a great security culture, while another with the same practices struggles and makes little headway.

Safety and security make us take more risks
We put a lot of work into safety and security. But how do we know any “best-practices” are actually making us more secure? Before you answer, you should consider that a number of studies have shown that drivers of vehicles with ABS tend to drive faster, follow more closely, and brake later, accounting for the failure of ABS to result in any measurable improvement in road safety. So we are getting places faster without improving our safety.

This counter-intuitive outcome is called “Risk Compensation” or “Risk Homeostasis.” It turns out that people seem to be inherently wired with a certain level of risk tolerance.  Study after study supports this idea. Bike helmets have been show to make cyclists ride faster, even increasing the fatality rate in some studies. Seat belts make you drive less carefully. Ski helmets result in more aggressive skiing. Safer skydiving equipment, children’s toys, even condoms -- all show an almost intentional desire to bring the accident and fatality rates back to their original levels.

People’s perception is obviously critically important to changing behavior. What do you notice about the pedestrians in the picture below?

Did you notice that those people are walking around without any traffic signs, warning, road markings, curbs, and crosswalks? Security madness! If this were an audit, their area would be shut down and sent for remediation. However, this “shared space” was consciously designed to increase the level of uncertainty for drivers and other road users. Amazingly, this approach has been found to result in lower vehicle speeds and fewer road casualties.

The perception of protection
Wait… What!? Protection technology makes people take more risks? Removing safety markings makes roads safer? Yep. You have to consider the human factor when you’re designing security interventions. Your best intentions could have completely opposite and unintended consequences. For example, maybe your new web application firewall gives developers a false sense of security and they stop doing input validation. Or maybe you rely on automated security testing that has so many false alarms that people start to ignore the results.

These problems start when the perception of security isn’t in balance with the reality of security protections. When perception is greater than the protection, people have a false sense of security and take unnecessary risks. On the other hand, when the protection exceeds the perception, the business will shy away from profitable activities.

Creating a culture of accelerated security evolution with transparency
Ultimately, we all want to achieve that elusive culture that makes security a part of information technology without everybody ending up mad. I think we can all agree that building stuff and trying to make it secure later isn’t the right approach. But neither is blindly following a process model that just aggregates a bunch of guesses about what might work.

The path to security that works is rapid evolution – the “builders” and “breakers” working to push security forward. The key to speeding up this process is to make security transparent, so that the perception of security matches up with the reality. The keys to transparency are:

  • Starting with a model. You have to start with an “expected model” of what you think your defenses should be. It doesn’t have to be perfect, as your model will evolve over time.
  • Getting coverage. Verify your expected model across all your applications to an appropriate level of rigor. Establish a security sensor network that can monitor your entire portfolio.
  • Getting continuous. Security visibility has a terrifically short half-life. Today’s highly accelerated software processes demand real-time feedback to developers and other stakeholders.

If you have processes that aren’t demonstrably effective within this framework, cut them. You’ll get a more effective security program and probably save a lot of time and money. What are your techniques for creating a strong security culture? Let me know how you know it works in the comments! Good luck.

A pioneer in application security, Jeff Williams is the founder and CTO of Contrast Security, a revolutionary application security product that enhances software with the power to defend itself, check itself for vulnerabilities, and join a security command and control ...
View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
7/16/2014 | 6:08:32 PM
Re: "shared space" philosophy versus reality
True.  But the point is that even though we aren't sure they help, we do all these security activities in the name of secure code.  I'm suggesting that we need to study the effectiveness of these activities because they *could* easily be undermining overall security.  It's not science to just "do what everyone else is doing" -- that ends up with expensive programs that everyone hates (sound familiar?)
User Rank: Author
7/16/2014 | 6:05:42 PM
Re: Why Your Application Security Program May Backfire
Absolutely agree that taking a positive approach to security is easier, faster, and more accurate.  It's the key to getting continuous.
User Rank: Author
7/11/2014 | 9:51:33 AM
Re: Why Your Application Security Program May Backfire
Nice read @JeffWilliams.  I tend to agree that the "know-do" gap of what users know versus what they do largely contributes of what you're discussing.  This begs the questions do we need to improve our security awareness programs in an effort to reduce this gap?

Well if you look at bike helmets or vehicle ABS in the context of managing security risk, perhaps this stems from how we continue to use traditional security approaches in a very different threat landscape.

Going to your comment about "Removing safety markings makes roads safer" brings me back to the idea that the use of positive security is much more effective in reducing risks in today's technology world.
User Rank: Apprentice
7/6/2014 | 12:07:30 PM
"shared space" philosophy versus reality
There is growing evidence that initial claims for reduced traffic sppeds and greater pedestrian safety in "shared spaces" is overstated.

See: Shared space - research, policy and problems - http://eprints.uwe.ac.uk/17937/8/tran1200047h.pdf

There is evidence that pedestrians and bicyclists "feel" less safe and adopt avoidance behaviors, sometimes avoiding "shared spaces" altogether.  Many advocates for the blind and physically impaired raise strong voices against "shared spaces" as extremely dangerous places to navigate, again urging those on behalf of whom they advocate to avoid such spaces. 

Seems that "sharing" favors drivers more than pedestrians and other users.
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Threat from the Internetand What Your Organization Can Do About It
The Threat from the Internetand What Your Organization Can Do About It
This report describes some of the latest attacks and threats emanating from the Internet, as well as advice and tips on how your organization can mitigate those threats before they affect your business. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-08-04
An issue was discovered in RICOH Streamline NX Client Tool and RICOH Streamline NX PC Client that allows attackers to escalate local privileges.
PUBLISHED: 2020-08-04
The administrative interface of Cohesive Networks vns3:vpn appliances before version 4.11.1 is vulnerable to authenticated remote code execution leading to server compromise.
PUBLISHED: 2020-08-04
Cross-site request forgery (CSRF) vulnerability in [Calendar01] free edition ver1.0.0 and [Calendar02] free edition ver1.0.0 allows remote attackers to hijack the authentication of administrators via unspecified vectors.
PUBLISHED: 2020-08-04
[Calendar01], [Calendar02], [PKOBO-News01], [PKOBO-vote01], [Telop01], [Gallery01], [CalendarForm01], and [Link01] [Calendar01] free edition ver1.0.0, [Calendar02] free edition ver1.0.0, [PKOBO-News01] free edition ver1.0.3 and earlier, [PKOBO-vote01] free edition ver1.0.1 and earlier, [Telop01] fre...
PUBLISHED: 2020-08-04
Privilege escalation vulnerability in SKYSEA Client View Ver.12.200.12n to 15.210.05f allows an attacker to obtain unauthorized privileges and modify/obtain sensitive information or perform unintended operations via unspecified vectors.