In my previous blog entries, I wrote about data de-duplication (See, Pitch something worth talking about!, Coolest Product, Episode 2: Technology). I believe data de-duplication will have similar impact on the way we store, transfer, protect and manage the data, as Fibre Channel Storage Area Network (FC SAN) did. Data de-duplication is the single most important innovation in data storage since FC-SAN.
It is no coincidence that both technologies initially targeted backup & recovery and then expanded in to other areas. Backup & Recovery continues to be an important function but a very expensive proposition for the businesses.
FC-SAN helped consolidate the backup footprint by enabling sharing of expensive backup devices. It also improved backup performance by separating backup traffic from client network. The volume of stored information is increasing at a rapid pace, and the need to reduce the backup footprint and performance further has risen again.
Virtual Tape Libraries (VTL) and Disk to Disk (D2D) backups already have shown significant promise in increasing backup performance. Now, data de-duplication is helping solve these problems with excellent data reduction, reportedly anywhere from 20 - 50:1, in backup savesets size and the reduced transfer size for offsite copy.
So what's next for data de-duplication?