Deduplication has moved from a risky hard to explain technology to one that is almost expected by customers from a disk backup device. Next generation backup deduplication systems are going to require a new set of capabilities to make them more than just disk backup. They will have to integrate with the backup software, begin to provide power management, and there needs to be a greater focus on recovery performance.Deduplication can now be delivered as either stand alone hardware or as a module to the backup software. In general a hardware appliance should have the advantage of being more globally useful, meaning you can send backup data streams from a variety of backup sources. Not only specific backup software but also dump commands within applications. The challenge is that in most cases the software has no idea what is going on behind the scenes, including when that data is being replicated or how to use that replicated data. Software deduplication, delivered as part of a module within a backup software application, has the advantage of a tighter integration. In other words the backup application knows that data is being deduped and should be able to leverage that fact.

Hardware deduplication vendors though are quickly embracing available API sets to integrate with backup applications. The most notable example today is Symantec's OpenStorage API that we detailed a while back in "A Backup API". With a backup API in place, hardware vendors can integrate with the backup application to improve performance and allow operations like replication to be controlled through the backup GUI instead of it having to be a separate process controlled through the backup appliance GUI.

Simple integration is just the beginning. As we discussed in our recent article "Integrating Disk Backup With Backup Software" we are beginning to see backup hardware suppliers create specific modules that will increase their integration capabilities with software, even if that software does not have a formal API set. We are also seeing some vendors provide backup application specific modules that can increase performance by offloading some of the data that needs to be processed by the deduplication system. In essence dividing the load up between the backup server and the disk backup appliance. We expect to see this trend continue.

Another challenge with backup deduplication hardware is power and space efficiency. A shortcoming of disk when compared to tape is clearly one of power consumption and with some disk systems the amount of data center floor space they consume. While you can throw plenty of darts at tape, one thing that can be denied is its power or density. In our next entry we will cover what needs to be improved when it comes to data center efficiency.

Track us on Twitter: http://twitter.com/storageswiss

Subscribe to our RSS feed.

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Find Storage Switzerland's disclosure statement here.