Vulnerabilities / Threats

07:00 PM
Connect Directly

The State of Application Penetration Testing

Data from real-world pen tests shows configuration errors and cross-site scripting are the most commonly found vulnerabilities.

Misconfiguration ranks as the most common type of vulnerability discovered in real-world penetration tests, according to a newly published study.

In client engagements last year, pen-testing-as-a-service provider Cobalt found mostly misconfiguration, followed by cross-site scripting (XSS), authentication and session, exposure of sensitive data, and access control-type vulnerabilities in applications.

Finding flaws is one thing, but fixing them is another. Redirect and forward-type flaws sit unresolved the longest of any category, 41 days, while server-side request forgery, sensitive data exposure, SQL injection, and others, including business logic, get fixed most rapidly.

Caroline Wong, vice president of security strategy for Cobalt and co-author of the Pen Test Metrics Study, says misconfiguration flaws are a sign of the times. "What that tells me is that so much of security vulnerability comes not necessarily from the code we were writing, and actually may have to do with other software and infrastructure components our software depends on in order to run," she says.

"That says … organizations are making huge use of cloud services and relying on others to do their settings for them. Maybe it's something they consider to be someone else's problem, and maybe in the past they didn't depend on third parties so they didn't consider [those] security settings."

Application penetration testing, unlike vulnerability assessment, is not exactly standard practice for most organizations today. Pen testing traditionally was associated with network security, but with the emergence of secure software development lifecycle (SDL) programs, more organizations are starting to opt for a white-hat hack of their apps. "I see a lot of organizations do one application pen test a year because of PCI, or HIPAA, or a customer asking them for one," Wong says. But more organizations are starting to opt for app pen tests to "do the right thing" in their secure development practices now, she says.

The most common software security program typically includes developer training and a penetration test to get a pulse on the state of their applications' security. Wong says organizations launching appsec for the first time go with a pen test to get them started.

"What's the biggest bang for their buck to start their program? 'Show me and my organization if we have real security issues, and use that information to get to the next level to justify further investment,'" she explains. "I find that pen testing is a very common first step when they first start thinking about appsec."

While a vulnerability assessment scans for and identifies flaws, a penetration test goes deeper, manually exploiting vulnerabilities to see how an attacker could abuse them, for example.

Most organizations typically focus on vulnerability scanning. "But by and large there are pockets who do true pen testing. I think there's a larger segment that does quasi-pen testing and not full, in-depth testing," says Kevin Greene, a software assurance evangelist. "I'm not sure all folks understand" app pen testing, he notes.

Ideally, in addition to an SDL check, organizations should run regular vulnerability assessments and pen tests to get "wider coverage" of the attack surface, he says.

Gary McGraw, vice president of security technology at Synopsys, says app pen testing is employed by 87% of all firms involved in the BSIMM8 software security maturity study. BSIMM  (Build Security In Maturity Model) reports on how more than 100 major companies from a range of vertical markets measure up with their software security development lifecycles. "Pen testing is a good 'smoke test' which can help uncover major problems," McGraw says. "Automated pen testing is available as a service and can be used to cover an entire application portfolio."

But pen testing alone doesn't make a software security program, he notes. "Pen testing is the third most important software security practice after code review with a static analysis tool and architectural risk analysis," McGraw says.  

Not All Apps Getting Tested

Cobalt's study also includes new data from a survey of security, management, operations, developers, and DevOps specialists. Turns out most aren't pen testing all of their apps: just 24% say they pen test 67% to 100% of their apps, while 35% test one to 33% of their apps, and 31% say they test anywhere from 34% to 66% of their apps.

While the best practice is to pen test critical apps once every quarter, most (32%) of the respondents in the survey say they only do so annually; 16%, semiannually; 12%, quarterly; 12%, not at all yet; and 7%, more than five times a year.

More than 30% say they pen test their apps when they add a new feature or patch. Some 26% pen test their apps on an ad hoc basis, 25% at the time of a new release, and 22% when a customer requests it.

Some 46% say they would pen test more apps, but it's too expensive. That's a common deterrent, as well the expertise required for pen testing. "It requires a really skilled individual" with the expertise to know what to probe and attack, according to Greene.

And once the results come in from a pen test, they require action. "I have met organizations for whom the reason they don't do more pen tests is because they are still trying to figure out how to fix the results from their first pen test," Wong says.

"The biggest challenge of pen testing apps is finding the right people," which can be costly, she notes.

Related Content:


Black Hat Asia returns to Singapore with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier solutions and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Jackson Higgins is Executive Editor at She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
The Case for Integrating Physical Security & Cybersecurity
Paul Kurtz, CEO & Cofounder, TruSTAR Technology,  3/20/2018
A Look at Cybercrime's Banal Nature
Curtis Franklin Jr., Senior Editor at Dark Reading,  3/20/2018
City of Atlanta Hit with Ransomware Attack
Dark Reading Staff 3/23/2018
Register for Dark Reading Newsletters
White Papers
Current Issue
How to Cope with the IT Security Skills Shortage
Most enterprises don't have all the in-house skills they need to meet the rising threat from online attackers. Here are some tips on ways to beat the shortage.
Flash Poll
[Strategic Security Report] Navigating the Threat Intelligence Maze
[Strategic Security Report] Navigating the Threat Intelligence Maze
Most enterprises are using threat intel services, but many are still figuring out how to use the data they're collecting. In this Dark Reading survey we give you a look at what they're doing today - and where they hope to go.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2017-05-09
NScript in mpengine in Microsoft Malware Protection Engine with Engine Version before 1.1.13704.0, as used in Windows Defender and other products, allows remote attackers to execute arbitrary code or cause a denial of service (type confusion and application crash) via crafted JavaScript code within ...

Published: 2017-05-08
unixsocket.c in lxterminal through 0.3.0 insecurely uses /tmp for a socket file, allowing a local user to cause a denial of service (preventing terminal launch), or possibly have other impact (bypassing terminal access control).

Published: 2017-05-08
A privilege escalation vulnerability in Brocade Fibre Channel SAN products running Brocade Fabric OS (FOS) releases earlier than v7.4.1d and v8.0.1b could allow an authenticated attacker to elevate the privileges of user accounts accessing the system via command line interface. With affected version...

Published: 2017-05-08
Improper checks for unusual or exceptional conditions in Brocade NetIron 05.8.00 and later releases up to and including 06.1.00, when the Management Module is continuously scanned on port 22, may allow attackers to cause a denial of service (crash and reload) of the management module.

Published: 2017-05-08
Nextcloud Server before 11.0.3 is vulnerable to an inadequate escaping leading to a XSS vulnerability in the search module. To be exploitable a user has to write or paste malicious content into the search dialogue.