Data Moveage: How To Move Data And Live To Tell About It

In a previous entry I wrote about the importance of moving data from primary storage to another platform. The roadblock is how to move that data from expensive storage to secondary storage. The traditional approach of deploying an agent on every server that monitors all the files and then moves files that haven't been accessed to a lower class of storage hasn't worked well in the enterprise. There are a variety of reasons, but most of the issues are the deployment and management of that many agents, plus the challenge of leaving stub files (files that point to where the actual file was moved) and managing those files.There also are software applications or appliances that don't require an agent that will walk or crawl the file systems of servers in your environment to find files that meet a certain criteria, old age being one, and then move those files to an alternative class of storage. The challenge with this approach is that they are very slow in this crawling/auditing process and they still have to manage the stub files that are left behind. They also need unprecedented access to the network so they can walk the mounted file systems.

Both solutions can be made to work, and in some cases work well, but for a simpler approach I like to recommend either Automated Tiered Storage or File Virtualization, or both, in some cases.

Automated Tiered Storage (ATS), as far as I know, is only available from one vendor today, Compellent. They call it data progression. The technology moves data to different classes of storage at a block level. No software agents are needed, no file system scans, it just works. Because it is a block-level based technology, it even works on data that wouldn't typically be moved to less expensive storage. Databases, for example, show up as one big active file to most data movers; to an IDM system they are blocks of data, with some of those blocks very active, while others are not. The ones that are not active are moved to a lower storage class, like less expensive ATA disk.

ATS has the benefit of...well...working. It is simple, it is automatic, and it just works. The result is that people use it and see the benefits of the technology almost instantly.

The downside to ATS is that it requires a storage system that can support it. If you aren't ready to part with your existing system or if you need the ability to move data to a different type of storage that is designed for long-term retention, like a disk-based archive, then a File Virtualization Appliance is a good fit. Companies such as Attune and OnStor offer these type of products.

Think of file virtualization as a DNS server for files. I don't know what IP address yahoo.com is assigned to, but I know when I type yahoo.com in my Web browser it comes up. This is how file virtualization products work. When I request george.doc I don't need to know what file server it is on, it's just served up to me.

File virtualization has the benefit of being able to move between different storage platforms, for example, from a primary NAS to a data deduplication appliance. The installation is a little more intrusive because the systems have to sit in-band between the users and the storage. Similar to ATS, it just works as well.

Data movement, the inhibitor to leveraging lower-cost storage, is now simple to implement and manage. You can either invest in a storage platform that has it built in or implement file virtualization to move data across multiple platforms.

George Crump is founder of Storage Switzerland, an analyst firm focused on the virtualization and storage marketplaces. It provides strategic consulting and analysis to storage users, suppliers, and integrators. An industry veteran of more than 25 years, Crump has held engineering and sales positions at various IT industry manufacturers and integrators. Prior to Storage Switzerland, he was CTO at one of the nation's largest integrators.