Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

News

5/28/2010
10:54 AM
George Crump
George Crump
Commentary
50%
50%

The Roll Down Hill Effect Of Primary Storage Deduplication

The adoption rate of deduplication in primary storage has been relatively low so far in primary storage. There are concerns on user's minds about performance impact, data integrity and how much capacity savings they will see. Clearly each of these concerns need to be addressed. When it comes to capacity savings though, there is a key component of capacity savings that might get overlooked, the roll down hill effect of proper primary storage deduplication.

The adoption rate of deduplication in primary storage has been relatively low so far in primary storage. There are concerns on user's minds about performance impact, data integrity and how much capacity savings they will see. Clearly each of these concerns need to be addressed. When it comes to capacity savings though, there is a key component of capacity savings that might get overlooked, the roll down hill effect of proper primary storage deduplication.Thus far the big winner in deduplication has been the backup process. If you are doing weekly full backups then there is plenty of opportunity for redundant data and you can post some incredible efficiency gains. This is not the case, or at least should not be, in primary storage. With the exception of virtualization images its unlikely that you will be able to make double digit storage efficiency gains thanks to deduplication alone. If you see typical efficiency claims of 12X in backup deduplication, expect maybe 5X gain in primary storage deduplication.

If you stop there though your missing an important part of the picture, the roll down hill effect of primary storage deduplication. If, and that is an important if, your primary storage deduplication technology can keep the data in an optimized state throughout its entire life cycle then you can see tremendous residual value in primary storage deduplication. With primary storage deduplication snapshots, replication, clones, extra copies of data (just in case copies) all now come at near zero capacity cost. For example you can perform dumps of your database every ten minutes if you want to, deduplication will curtail the capacity growth that would normally create.

The key issue is if and when primary storage deduplication will need to "re-inflate" to a non-optimized data state. Optimization throughout the data lifecycle and the tiers of storage it is on, is critical for making deduplication make sense in primary storage. In fairness there may be a time you want to re-inflate on purpose and remove dependency on the deduplication hash table. That is going to depend on how much you trust your deduplication technology to maintain its meta-data and provide rich data integrity features.

Deduplication technology tries to fix the capacity explosion problem faced by most data centers. Where deduplication is being successful right now, in backup repositories, is trying to fix that problem after it has already occurred. Primary storage deduplication that maintains data in its optimized state fixes the problem before it becomes a problem. If properly implemented primary storage deduplication could have significant reduction on the storage demands of your data center.

Track us on Twitter: http://twitter.com/storageswiss

Subscribe to our RSS feed.

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Find Storage Switzerland's disclosure statement here.

 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
karthickkandaiyah2
50%
50%
karthickkandaiyah2,
User Rank: Apprentice
12/27/2012 | 4:01:16 PM
re: The Roll Down Hill Effect Of Primary Storage Deduplication
good one
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/3/2020
'BootHole' Vulnerability Exposes Secure Boot Devices to Attack
Kelly Sheridan, Staff Editor, Dark Reading,  7/29/2020
Average Cost of a Data Breach: $3.86 Million
Jai Vijayan, Contributing Writer,  7/29/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Threat from the Internetand What Your Organization Can Do About It
The Threat from the Internetand What Your Organization Can Do About It
This report describes some of the latest attacks and threats emanating from the Internet, as well as advice and tips on how your organization can mitigate those threats before they affect your business. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-13151
PUBLISHED: 2020-08-05
Aerospike Community Edition 4.9.0.5 allows for unauthenticated submission and execution of user-defined functions (UDFs), written in Lua, as part of a database query. It attempts to restrict code execution by disabling os.execute() calls, but this is insufficient. Anyone with network access can use ...
CVE-2017-18112
PUBLISHED: 2020-08-05
Affected versions of Atlassian Fisheye allow remote attackers to view the HTTP password of a repository via an Information Disclosure vulnerability in the logging feature. The affected versions are before version 4.8.3.
CVE-2020-15109
PUBLISHED: 2020-08-04
In solidus before versions 2.8.6, 2.9.6, and 2.10.2, there is an bility to change order address without triggering address validations. This vulnerability allows a malicious customer to craft request data with parameters that allow changing the address of the current order without changing the shipm...
CVE-2020-16847
PUBLISHED: 2020-08-04
Extreme Analytics in Extreme Management Center before 8.5.0.169 allows unauthenticated reflected XSS via a parameter in a GET request, aka CFD-4887.
CVE-2020-15135
PUBLISHED: 2020-08-04
save-server (npm package) before version 1.05 is affected by a CSRF vulnerability, as there is no CSRF mitigation (Tokens etc.). The fix introduced in version version 1.05 unintentionally breaks uploading so version v1.0.7 is the fixed version. This is patched by implementing Double submit. The CSRF...