Reduce Storage with Windows Server 2012 Deduplication

Deduplication is a new feature for File Services Role in Windows Server 2012 that helps to reduce the required storage space. This is done by inspecting the stored data. Identical parts are replaced by only a reference to a single stored part. Deduplication works great for data that has similar parts and is rarely modified, e.g. Archives. However, Deduplication works only for volumes. Migration to an iSCSI connected LUN is an option to get a deduplicatable volume.

• Select a Windows Server 2012 machine to serve as Deduplicator
• Open your storage system configuration interface and create a new iSCSI target and LUN
• Start the Windows builtin iSCSI Initiator, connect the LUN, init and format the volume
• Copy all files from the traditional share to the connected iSCSI LUN

Here is an example for an iSCSI connected LUN. Drive S: is our Software Archive with installation files, MSDN Downloads, Trial Versions, etc. All these files require about 480 GB storage.

• Open Server Manager and select the File Service Roles
• Navigate to the volume to deduplicate, open the context menu to configure the Deduplication schedule
• In the schedule dialog configure if and when the Deduplication should run in background
  If you’ve just moved files from a file share to a connected LUN be aware that the modified timestamp was updated and Deduplication might not immediately start Deduplication. However, this can be configured in the Deduplication schedule dialog.

Here is the Deduplication result applied to the software archive. Windows came up with a Deduplication rate about 58% and required storage was reduced by 285 GB.

• Finally create new file share on the Windows Server Machine in order to grant access over the network

Update 28.4: Deduplication Applied to HyperV Image Template Archive achieved 79% deduplication