02:09 PM
Connect Directly

Flat-File Databases Often Overlooked In Security Schemes

Popular method of creating and exchanging database files could leave sensitive data vulnerable, experts say

While many enterprises continue to build systems and strategies to protect their relational databases, many organizations haven't done enough to secure the information in the most common database format: the flat-file database, experts say.

It's a dirty little secret of data protection" Most enterprises have sensitive data stored in flat files and floating around the network. Many of these files remain unencrypted and unprotected by the stringent controls that are typically applied to relational databases.

"If you asked a group of IT people, 'Can you think of a flat-file database on your network?,' most people would say, 'Yeah, they're all over the place,'" says Rob Ayoub, global program director of network security for Frost & Sullivan. "But ask, 'How do you protect that data?,' and you might not get an answer.

"We've gotten very good at protecting the critical data within our main databases. But how many times have organizations had employees pull subsets of that data -- without wondering how that data is stored and handled?"

Flat files are a way of life for organizations that need to manipulate data on different systems, migrate data between applications and platforms (think mainframes and COBOL), and even push files across different database platforms, says David Friedland, vice president of business development for Innovative Routines International (IRI).

IRI is an independent software vendor that specializes in data manipulation software -- and, more recently, security products for flat files. "They're just ubiquitous," Friedland says. "Typically, flat files are involved when you're doing large extracts or bulk loads; it's really for very high volume databases and data staging within data warehouses. They call it the 'sequential file stage' in the data transformation world.

"But that's just one way in which flat files are used. Other people just have them around because they feed spreadsheets, or they come out of mainframe applications."

But flat-file data stores are usually less secure than their relational counterparts, experts agree.

"The biggest kind of risk is usually that the implementations are generally not handled well," says Rich Mogull, analyst for Securosis. "Frequently, it will be an application programmer who has just kind of built their own little thing -- as opposed to using an actual managed system -- and they basically have built pretty poor, if any, security around that.

"With a regular database, there's at least some level of security built in. You're, of course, going to have authentication and authorization because those are core elements of the platform. With flat files, you may or may not get that."

Flat files are also a challenge because they move quickly around the network, but can be difficult to locate.

"I think that's one of the real weaknesses of flat files -- they're not transparent, you don't know as much about the data as you do about data that is in another kind of system," says David Stodder, analyst for Perceptive Information Strategies and Ventana Research. "Flat files tend to be the type of thing where organizations will take pieces of it and move it into some other data source -- for example, a data manipulation engine. That in itself is a challenge, just being able to maintain the security as it is traveling around the organization."

Mogull says that as organizations continue their efforts to comply with PCI mandates, many of them are starting to recognize that flat files could be the Achilles' heel of their strategies for protecting personally identifiable information (PII). If the flat file is not part of a managed database solution, then organizations must find ways to encrypt PII in flat files, he advises.

"For sensitive data, you will, of course, need to encrypt that," Mogull says. "If it's not part of your managed database solution, then it's something that you're going to have to go ahead and handle yourself using a crypto toolkit or something else.

"And, boy, I really recommend people be cautious and avoid doing that if they can. Because one of the biggest problems is if someone tries to doing their own crypto -- even using a toolkit -- frequently, they'll screw it up."

This problem is the reason that IRI recently introduced its own field-level encryption solution, FieldShield, which is designed specifically for flat files, Friedland says. However, encryption is only part of the flat-file security solution. Before you can encrypt sensitive information, you'll need to find a way to discover flat files across the infrastructure -- and then find the sensitive information within those files that needs encryption.

"I think the discovery is a huge part of the flat files' dilemma," Ayoub says. "You get a programmer who is working with some old data, dumps the data into a flat file just for testing, and then no one else knows it's out there. There are just huge issues with the tracking and finding of data, and I think discovery is probably an even bigger problem for flat files [than it is for] object-oriented databases." Like several other experts, Ayoub believes organizations should not only implement technological controls to protect data within flat-file databases, but they also need better governance over when, how, and who creates these data dumps.

"In a lot of organizations the policies around data-handling may not be very well defined," Ayoub says. "Some have become more savvy and have set up formal polices around the kind of live data you can dump. But I think it's something that not a lot of people think about."

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message. Ericka Chickowski specializes in coverage of information technology and business innovation. She has focused on information security for the better part of a decade and regularly writes about the security industry as a contributor to Dark Reading.  View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Register for Dark Reading Newsletters
White Papers
Current Issue
Security Operations and IT Operations: Finding the Path to Collaboration
A wide gulf has emerged between SOC and NOC teams that's keeping both of them from assuring the confidentiality, integrity, and availability of IT systems. Here's how experts think it should be bridged.
Flash Poll
New Best Practices for Secure App Development
New Best Practices for Secure App Development
The transition from DevOps to SecDevOps is combining with the move toward cloud computing to create new challenges - and new opportunities - for the information security team. Download this report, to learn about the new best practices for secure application development.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio
Archived Dark Reading Radio
In past years, security researchers have discovered ways to hack cars, medical devices, automated teller machines, and many other targets. Dark Reading Executive Editor Kelly Jackson Higgins hosts researcher Samy Kamkar and Levi Gundert, vice president of threat intelligence at Recorded Future, to discuss some of 2016's most unusual and creative hacks by white hats, and what these new vulnerabilities might mean for the coming year.