dm-cache (Device Mapper cache) is a device mapper target to improve performance of a block device (e.g., a spindle device ) HDD by dynamically migrating some of its data to a faster, smaller device (SSD). NVMe based PCIe SSD will act as the cache device. 
For more details about the NVMe based PCIe SSD refer the blog in the link
http://en.community.dell.com/techcenter/os-applications/w/wiki/7581.nvme-on-rhel7
dm-cache can be configured using the user space tool set LVM2( logical volume manager tools).

Virtual cache created by dm-cache is made of three physical devices. (Refer Fig1).The Origin device is the slower destination storage device  (HDD).The Cache device is used to store the user data block and the meta device stores the meta data like block placement, dirty flags and other internal data. In this case we use same device [Cache device] for both data blocks and metadata

Figure 1 : Steps to setup and configure dm-cache using LVM tools

 

Hardware Setup:
Origin Device [HDD] : /dev/sda1
Cache Device [NVMe based PCIe SSD] (Block/Meta): /dev/nvme0n1p1
 
Software Setup:
Operating system : Rhel 7
Tools : LVM2 user space tools for dm-cache configuration
 

Configuration Steps:

Following are the steps to configure the Virtual device dm-cache using LVM .

Each step output maps to block with numbering mentioned in Fig.1

1)    Create 100 GB partition on the hard disk [Origin Device]

parted -a optimal /dev/sda mkpart primary 1 100G

This creates /dev/sda1

2)    Create 10 GB partition on the NVMe PCIe SSD disk [Cache Device]

parted -a optimal /dev/nvme0n1 mkpart primary 1 10G

3)    Create a Volume group (cache) with the Origin and Cache device

vgcreate cache /dev/sda1  /dev/nvme0n1p1

Check for the “cache” volume group by vgdisplay

[root@localhost ~]# vgdisplay

--- Volume group ---

VG Name               cache

System ID

Format                lvm2

Metadata Areas        2

Metadata Sequence No  7

VG Access             read/write

VG Status             resizable

MAX LV                0

Cur LV                2

Open LV               0

Max PV                0

Cur PV                2

Act PV                2

VG Size               103.36 GiB

PE Size               4.00 MiB

Total PE              26461

Alloc PE / Size       25399 / 99.21 GiB

Free  PE / Size       1062 / 4.15 GiB

VG UUID               Zd8dNe-6Kdt-7qgY-dmSN-8WHe-4wqQ-euM3Ql

 

4)    Create  Origin Device logical Volume (origin_device)

lvcreate -l 90%FREE -n origin_device cache /dev/sda1

5)    Create a cache meta data logical Volume  (cache_meta)

lvcreate -l 5%FREE -n cache_meta cache /dev/nvme0n1p1

6)    Create a cache  block data logical Volume  (cache_block)

lvcreate -l 80%FREE -n cache_block cache /dev/nvme0n1p1

7)    Create a cache pool logical volume by combining cache block and cache meta data logical volumes (combining step 5 and 6)

lvconvert --type cache-pool --poolmetadata cache/cache_meta cache/cache_block

NOTE : The created cache pool will have the same name as “cache_block” that is the name of cache  block data logical volume created in step 6.

This is the behavior of the tool

Check for the “cache” volume group by lvdisplay

root@localhost ~]# lvdisplay

--- Logical volume ---

LV Path                /dev/cache/cache_block

LV Name                cache_block

VG Name                cache

LV UUID                kWYQxP-Jdlr-JdxE-aleB-JJpj-3rmw-Q0cojx

LV Write Access        read/write

LV Creation host, time localhost.localdomain, 2014-06-28 09:05:32 -0400

LV Status              available

# open                 0

LV Size                5.07 GiB

Current LE             1297

Segments               1

Allocation             inherit

Read ahead sectors     auto

- currently set to     256

Block device           253:2

 

 

8)    Create a cache logical volume by linking the cache pool (step 7) logical volume and origin_device logical volume (step 4)

lvconvert --type cache --cachepool cache/cache_block cache/origin_device

This creates the virtual cache device (dm-cache)  orign_device as shown below as dm4

[root@localhost ~]# ls -l /dev/cache/origin_device

lrwxrwxrwx. 1 root root 7 Jun 28 09:13 /dev/cache/origin_device -> ../dm-4

[root@localhost ~]#

 

9)    Format the Virtual cache device with a file system and use it

mkfs.ext4 /dev/cache/origin_device

Performance data  

Performance run done using fio tool shows a great improvement of the read,write and read-write mix operations.

Table-1 shows the FIO tool output of performance difference and the improvement in percentage with the dmcache in place  compared to the normal drive (rotational drive) and Fig 2.shows the corresponding graph plot.

The test is done with 1M block size for Random Read,Random Write and Random Read-Write mix (50%) on the Original hard drive(sda) and the drive with Dmcache configured

FIO at BS = 1M

IOPS

DEVICE

Read

Write

Read-Write

Drive with dmcache

197

798

159

sda [original drive]

124

118

58

Improvement in %

58.87%

576.20%

174.13%

 Table .1 .Performance Improvement with Dmcache

 

 Fig 2 .Performance Improvement with Dmcache

                                                                                                                             

Conclusion : Thus  NVMe PCIe SSD based dm-cache improves the  performance of legacy spindle based hard drives to a greater extent