Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


08:00 AM
Connect Directly

Real Data in App Testing Poses Real Risks

Simulated or 'anonymized' data is a better option than exposing live data to outside sources

If you use real, live customer data in your testing and development of applications, you may want to think twice about the risks of exposing that data.

Organizations that use live data in their testing do so basically because it makes the testing more real-world and better puts the app through its paces. Trouble is, it also can expose sensitive data to engineering staff who normally wouldn't have access to that data, as well as to consultants and other outside contractors working with your organization on the testing process.

But you don't have to use the real thing in app testing and development: "It needs to be real enough, but it's better if it's not people's confidential information," says Gary McGraw, CTO of Cigital.

Still, it's common practice among many organizations today. According to a new study from the Ponemon Institute, which was commissioned by Compuware, 69 percent of the over 800 IT professionals surveyed said they use live data for testing their applications, and 62 percent say they do so in their software development. Over 50 percent outsource their app testing, and of that group, 49 percent of them share live data with the outsourcing organization.

"This flies under the radar. It's actually a common practice, although I don't know any statistics on it," says Chris Eng, director of security research for Veracode. "When we're doing penetration testing of an application… we find that, yes, those testing environment databases were just copied from a production/live database," for example.

But compliance and other pressures are pushing some organizations to reassess that practice, Eng says. "The minute you stick production data into a test environment, you're suddenly exposing it to everyone in the organization and quality assurance, as well as to any consultants or external contactors local or offshore. You're now widening the net and exposing it to a lot of parties that shouldn't have access to it."

One option is to develop simulated data that looks a lot like the real thing, notes Cigital's McGraw. His firm did so for the Financial Industry Regulatory Authority (FINRA) during a transaction application testing process. "It requires writing a little code," he notes. For the FINRA app, that meant ensuring the key trade elements were there -- dollar amounts and valid stock ticker symbols, for instance -- so it appeared as close the real thing as possible.

In the Ponemon study, 89 percent of the organizations running live data in their testing use customer files, and 74 percent, customer lists. Among the live data: employee and vendor records, customer account numbers, credit card numbers, Social Security numbers, and credit, debit, and payment information.

The study points to one case last year where an outside consultant hired by an insurance firm to develop applications turned around and sold some of the firm's customer data he had been privy to during the development project.

Meanwhile, Veracode's Eng says the problem with fake data is that it can sometimes compromise testing. Even something as minor as a missing punctuation mark can throw things off. "An apostrophe isn't something you'd think was a big deal, but it's important in SQL queries, for example," he says. "If you don't have a realistic set of data it may not be possible to catch all the SQL injection issues."

"Anonymizing" the data is an effective option, he says. "It boils down to taking the real data and masking it and transforming different parts that make it no longer identifiable," he says. "It's no longer the real data, but the structure of the data is the same."

But securing live data in testing isn't a priority for most organizations, says Paul Vallely, solutions sales director for enterprise solutions at Compuware. "All the organizations we interact with say this is a risk, but it's not being prioritized enough. They have so many projects that they need to deliver" that it gets lost in the shuffle.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.

  • Cigital Inc.
  • Compuware Corp. (Nasdaq: CPWR)
  • Ponemon Institute LLC
  • Veracode

    Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

    Recommended Reading:

    Comment  | 
    Print  | 
    More Insights
  • Comments
    Newest First  |  Oldest First  |  Threaded View
    COVID-19: Latest Security News & Commentary
    Dark Reading Staff 8/3/2020
    Pen Testers Who Got Arrested Doing Their Jobs Tell All
    Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
    'BootHole' Vulnerability Exposes Secure Boot Devices to Attack
    Kelly Sheridan, Staff Editor, Dark Reading,  7/29/2020
    Register for Dark Reading Newsletters
    White Papers
    Cartoon Contest
    Current Issue
    Special Report: Computing's New Normal, a Dark Reading Perspective
    This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
    Flash Poll
    The Changing Face of Threat Intelligence
    The Changing Face of Threat Intelligence
    This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
    Twitter Feed
    Dark Reading - Bug Report
    Bug Report
    Enterprise Vulnerabilities
    From DHS/US-CERT's National Vulnerability Database
    PUBLISHED: 2020-08-05
    LimeSurvey 4.3.2 allows reflected XSS because application/controllers/LSBaseController.php lacks code to validate parameters.
    PUBLISHED: 2020-08-05
    USVN (aka User-friendly SVN) before 1.0.9 allows XSS via SVN logs.
    PUBLISHED: 2020-08-05
    IBM UrbanCode Deploy (UCD),,, and is vulnerable to an XML External Entity Injection (XXE) attack when processing XML data. A remote attacker could exploit this vulnerability to expose sensitive information or consume memory resources. IBM X-Force ID: 181848.
    PUBLISHED: 2020-08-05
    CAMS for HIS CENTUM CS 3000 (includes CENTUM CS 3000 Small) R3.08.10 to R3.09.50, CENTUM VP (includes CENTUM VP Small, Basic) R4.01.00 to R6.07.00, B/M9000CS R5.04.01 to R5.05.01, and B/M9000 VP R6.01.01 to R8.03.01 allows a remote unauthenticated attacker to bypass authentication and send altered c...
    PUBLISHED: 2020-08-05
    Directory traversal vulnerability in CAMS for HIS CENTUM CS 3000 (includes CENTUM CS 3000 Small) R3.08.10 to R3.09.50, CENTUM VP (includes CENTUM VP Small, Basic) R4.01.00 to R6.07.00, B/M9000CS R5.04.01 to R5.05.01, and B/M9000 VP R6.01.01 to R8.03.01 allows a remote unauthenticated attacker to cre...