Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


02:09 PM
Connect Directly

Flat-File Databases Often Overlooked In Security Schemes

Popular method of creating and exchanging database files could leave sensitive data vulnerable, experts say

While many enterprises continue to build systems and strategies to protect their relational databases, many organizations haven't done enough to secure the information in the most common database format: the flat-file database, experts say.

It's a dirty little secret of data protection" Most enterprises have sensitive data stored in flat files and floating around the network. Many of these files remain unencrypted and unprotected by the stringent controls that are typically applied to relational databases.

"If you asked a group of IT people, 'Can you think of a flat-file database on your network?,' most people would say, 'Yeah, they're all over the place,'" says Rob Ayoub, global program director of network security for Frost & Sullivan. "But ask, 'How do you protect that data?,' and you might not get an answer.

"We've gotten very good at protecting the critical data within our main databases. But how many times have organizations had employees pull subsets of that data -- without wondering how that data is stored and handled?"

Flat files are a way of life for organizations that need to manipulate data on different systems, migrate data between applications and platforms (think mainframes and COBOL), and even push files across different database platforms, says David Friedland, vice president of business development for Innovative Routines International (IRI).

IRI is an independent software vendor that specializes in data manipulation software -- and, more recently, security products for flat files. "They're just ubiquitous," Friedland says. "Typically, flat files are involved when you're doing large extracts or bulk loads; it's really for very high volume databases and data staging within data warehouses. They call it the 'sequential file stage' in the data transformation world.

"But that's just one way in which flat files are used. Other people just have them around because they feed spreadsheets, or they come out of mainframe applications."

But flat-file data stores are usually less secure than their relational counterparts, experts agree.

"The biggest kind of risk is usually that the implementations are generally not handled well," says Rich Mogull, analyst for Securosis. "Frequently, it will be an application programmer who has just kind of built their own little thing -- as opposed to using an actual managed system -- and they basically have built pretty poor, if any, security around that.

"With a regular database, there's at least some level of security built in. You're, of course, going to have authentication and authorization because those are core elements of the platform. With flat files, you may or may not get that."

Flat files are also a challenge because they move quickly around the network, but can be difficult to locate.

"I think that's one of the real weaknesses of flat files -- they're not transparent, you don't know as much about the data as you do about data that is in another kind of system," says David Stodder, analyst for Perceptive Information Strategies and Ventana Research. "Flat files tend to be the type of thing where organizations will take pieces of it and move it into some other data source -- for example, a data manipulation engine. That in itself is a challenge, just being able to maintain the security as it is traveling around the organization."

Mogull says that as organizations continue their efforts to comply with PCI mandates, many of them are starting to recognize that flat files could be the Achilles' heel of their strategies for protecting personally identifiable information (PII). If the flat file is not part of a managed database solution, then organizations must find ways to encrypt PII in flat files, he advises.

"For sensitive data, you will, of course, need to encrypt that," Mogull says. "If it's not part of your managed database solution, then it's something that you're going to have to go ahead and handle yourself using a crypto toolkit or something else.

"And, boy, I really recommend people be cautious and avoid doing that if they can. Because one of the biggest problems is if someone tries to doing their own crypto -- even using a toolkit -- frequently, they'll screw it up."

This problem is the reason that IRI recently introduced its own field-level encryption solution, FieldShield, which is designed specifically for flat files, Friedland says. However, encryption is only part of the flat-file security solution. Before you can encrypt sensitive information, you'll need to find a way to discover flat files across the infrastructure -- and then find the sensitive information within those files that needs encryption.

"I think the discovery is a huge part of the flat files' dilemma," Ayoub says. "You get a programmer who is working with some old data, dumps the data into a flat file just for testing, and then no one else knows it's out there. There are just huge issues with the tracking and finding of data, and I think discovery is probably an even bigger problem for flat files [than it is for] object-oriented databases." Like several other experts, Ayoub believes organizations should not only implement technological controls to protect data within flat-file databases, but they also need better governance over when, how, and who creates these data dumps.

"In a lot of organizations the policies around data-handling may not be very well defined," Ayoub says. "Some have become more savvy and have set up formal polices around the kind of live data you can dump. But I think it's something that not a lot of people think about."

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message. Ericka Chickowski specializes in coverage of information technology and business innovation. She has focused on information security for the better part of a decade and regularly writes about the security industry as a contributor to Dark Reading.  View Full Bio


Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/10/2020
Pen Testers Who Got Arrested Doing Their Jobs Tell All
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
Researcher Finds New Office Macro Attacks for MacOS
Curtis Franklin Jr., Senior Editor at Dark Reading,  8/7/2020
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-08-10
A cross-site scripting (XSS) vulnerability in the Credential Manager component in SAINT Security Suite 8.0 through 9.8.20 could allow arbitrary script to run in the context of a logged-in user when the user clicks on a specially crafted link.
PUBLISHED: 2020-08-10
An SQL injection vulnerability in the Assets component of SAINT Security Suite 8.0 through 9.8.20 allows a remote, authenticated attacker to gain unauthorized access to the database.
PUBLISHED: 2020-08-10
An SQL injection vulnerability in the Analytics component of SAINT Security Suite 8.0 through 9.8.20 allows a remote, authenticated attacker to gain unauthorized access to the database.
PUBLISHED: 2020-08-10
A cross-site scripting (XSS) vulnerability in the Permissions component in SAINT Security Suite 8.0 through 9.8.20 could allow arbitrary script to run in the context of a logged-in user when the user clicks on a specially crafted link.
PUBLISHED: 2020-08-10
In MyBB before version 1.8.24, the custom MyCode (BBCode) for the visual editor doesn't escape input properly when rendering HTML, resulting in a DOM-based XSS vulnerability. The weakness can be exploited by pointing a victim to a page where the visual editor is active (e.g. as a post or Private Mes...