{"id":4012,"date":"2020-08-05T10:24:26","date_gmt":"2020-08-05T14:24:26","guid":{"rendered":"https:\/\/solutionsreview.com\/backup-disaster-recovery\/?p=4012"},"modified":"2022-07-05T09:13:19","modified_gmt":"2022-07-05T13:13:19","slug":"the-pros-and-cons-of-deduplication-appliances-with-cobalt-iron","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/backup-disaster-recovery\/the-pros-and-cons-of-deduplication-appliances-with-cobalt-iron\/","title":{"rendered":"The Pros and Cons of Deduplication Appliances with Cobalt Iron"},"content":{"rendered":"

Solutions Review spoke with Douglas Chando, solutions architect at Cobalt Iron<\/a> about the pros and cons of using deduplication appliances for backup.<\/i><\/b><\/p>\n

For people who aren\u2019t aware, what is a deduplication appliance?<\/b><\/h4>\n

A deduplication appliance is a device that was intended to enable organizations to eliminate their dependency on tape by more efficiently writing data, or backup copies, to disk. One of the more well-known deduplication appliances in the market is Data Domain, and their tagline for one of their original campaigns was \u201cTape sucks\u201d.\u00a0<\/span><\/p>\n

Data Domain,<\/span><\/a> as well as any other deduplication appliance, is a pre-configured device that combines compute, storage, and an OS. It is purpose-built for ingesting data, finding commonality in the blocks of data, and reducing the common data to only a single instance, thereby reducing the storage footprint. One of the more common use cases for these appliances was\/is backup storage repositories, and due to the duplicity of the nature of backups, deduplication had a huge impact on the efficiency in how backup data is stored.<\/span><\/p>\n

\t\t\t

<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div><\/span><\/p>\n

How does deduplication work for backup data?<\/b><\/h4>\n

Great question. The purpose of deduplication devices is to take blocks of data, compare it to other blocks of data on the device, and find common blocks so that redundant data can be reduced. This is done by using either an SHA or MD algorithm. While this was a great approach when deduplication initially came out over 15 years ago, the downside is that this process can be very resource-intensive. Dedicating hardware resources to the process of deduplication ensured repeatable and predictable performance – but technology has moved on quite significantly over the years.\u00a0<\/span><\/p>\n

At their advent, these deduplication appliances did what is commonly referred to as post-process deduplication, meaning that the data was sent to the device and was deduplicated afterward. Although deduplication did a great job in reducing the data that was stored on the device, it meant that redundant data flooded the network on a routine basis.\u00a0<\/span><\/p>\n

Can you give an example?<\/b><\/h4>\n

\u00a0Sure thing. To illustrate, let\u2019s say every night when a backup would run it would send its full data set of 500GB. However, when the data reached the dedupe device, it would only write the 10MB of changes that might have occurred. This wouldn\u2019t exactly be ideal because of how long the process takes, and how taxing this can be on a network when the same thing is occurring across many systems.<\/span><\/p>\n

As time went on, this network inefficiency became a real problem, and with data increasing exponentially, it wasn\u2019t something that was going away. To address this network inefficiency, Data Domain introduced a technology called Boost, which was software that resided on a client that would essentially look at data that was being sent before it was sent to the appliance. If it was a duplicate, it would not send it \u2013 this is what is commonly called client-side or in-line deduplication.\u00a0<\/span><\/p>\n

While it was a needed enhancement to deduplication technology, the question soon arose from a budgeting standpoint: \u201cIf I am using the compute resources on my server to perform deduplication, why do I need a costly purpose-built appliance to do this?\u201d Because of the high cost, combined with many backup solutions having a form of software-based deduplication, data professionals have started to question if a dedicated deduplication appliance is still necessary.<\/span><\/p>\n

With what you have pointed out, why do you believe dedicated deduplication appliances are still in such heavy use?<\/b><\/h4>\n

Initially, the advantages of using a deduplication device were very obvious. Organizations were looking to get away from tape backup technologies, but writing all of their data to disk was cost-prohibitive. Deduplication enabled them to consume a fraction of the disk, thereby affording them the opportunity to consider a disk-based backup strategy. However, with advancements in technology, the benefits of a dedicated appliance specifically for the purpose of deduplication have diminished significantly.\u00a0<\/span><\/p>\n

I attribute the demand of dedicated deduplication appliances to legacy solutions that have it heavily embedded into an organization\u2019s infrastructure, combined with diehard advocates of these appliances. Many organizations continue to use deduplication devices because they follow the old adage, \u201cif it ain\u2019t broke, don\u2019t fix it\u201d. However, as discussed earlier, data size increases will push this aging technology to its limits, and the corrective action of refreshing the technology will make it cost-prohibitive to continue the course.<\/span><\/p>\n

Other than what you have already mentioned, what are the disadvantages of using dedicated deduplication?<\/b><\/h4>\n

Here is what I consider the main disadvantages:<\/span><\/p>\n