How to configure multipath on enterprise linux 6.x for Dell compellent storage?

Applies to:

-------------------------------------------------------------------------------------

Operating systems: Oracle Linux 6.x, RHEL 6.x

Storage: Dell Compellent SC040 & SC8000 with Storage Center 6.3+

Problem:

-------------------------------------------------------------------------------------

How to configure multipath on Enterprise Linux 6 for Dell Compellent Storage?

Solution:

-------------------------------------------------------------------------------------

This article provides a description, along with best practices, on how to configure multipath devices on Enterprise Linux 6.x for Dell Compellent Storage Center. This includes:

• How to discover Compellent multipath volumes?
• How to configure the Compellent multipath devices using device-mapper?
• Recommended host bus adapter (HBA) settings

Setup the Compellent Storage as per the Compellent SAN Connectivity guide which includes the best practices to configure the controllers, enclosures, front end and the backend HBA ports. The CT-SC040 & SC8000 Connectivity Guide can be accessed at http://kc.compellent.com

Discovering Compellent multipath volumes:

This section helps to determine the Storage Center volume that correlates to a specific Linux device. The following Linux commands and tools can be used to discover and identify the Compellent volumes:

1. Discovering the new volumes: Discover a volume from the host by scanning the Fibre Channel (FC) HBA’s using the below command:

# echo "- - -" >> /sys/class/scsi_host/host0/scan

 

The discovery information of new volumes can be seen in dmesg output as below.

SCSI device sdf: 587202560 512-byte hdwr sectors (300648 MB) sdf: Write Protect is off sdf: Mode Sense: 87 00 00 00 SCSI device sdf: drive cache: write through SCSI device sdf: 587202560 512-byte hdwr sectors (300648 MB) sdf: Write Protect is off sdf: Mode Sense: 87 00 00 00 SCSI device sdf: drive cache: write through sdf: unknown partition table sd 0:0:3:15: Attached scsi disk sdf sd 0:0:3:15: Attached scsi generic sg13 type 0

2. Identifying the volumes: Identify the Compellent Storage volumes and correlate to a specific Linux device using the tools below.

lsscsi: lsscsi is a tool that parses information from the /proc and /sys virtual file systems into a simple human readable output. This tool is present in the Linux 6 base repository which can be installed using the Yum package manager.

# lsscsi [0:2:0:0] disk DELL PERC H710 3.13 /dev/sda [7:0:0:1] disk COMPELNT Compellent Vol 0603 /dev/sdb [7:0:1:1] disk COMPELNT Compellent Vol 0603 /dev/sdc [7:0:2:2] disk COMPELNT Compellent Vol 0603 /dev/sdd [7:0:3:2] disk COMPELNT Compellent Vol 0603 /dev/sde [8:0:0:1] disk COMPELNT Compellent Vol 0603 /dev/sdf [8:0:1:1] disk COMPELNT Compellent Vol 0603 /dev/sdg [8:0:2:2] disk COMPELNT Compellent Vol 0603 /dev/sdh [8:0:3:2] disk COMPELNT Compellent Vol 0603 /dev/sdi

The above output shows eight drives from the Storage Center. The first column above shows the [host:channel:target:lun] designation for the volume. The first number corresponds to the local HBA hostX that the volume is mapped to. Channel is the SCSI bus address which will always be zero. The third number correlates to the Storage Center front end ports (targets). The last number is the LUN that the volume is mapped on.

/proc/scsi/scsi: The luns and targets information can also be found in /proc/scsi/scsi file if lsscsi is not installed.

# cat /proc/scsi/scsi Host: scsi0 Channel: 02 Id: 00 Lun: 00 Vendor: DELL Model: PERC H710 Rev: 3.13 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi7 Channel: 00 Id: 00 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0603 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi7 Channel: 00 Id: 01 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0603 Type: Direct-Access ANSI SCSI revision: 05

scsi_id: scsi_id can be used to report the World Wide Identifier (WWID) of a volume and is available in all base installations. This WWID can be matched to the volume serial number reported in the Storage Center GUI for accurate correlation.

# scsi_id -g -u /block/sdd 36000d310000065000000000000000050

Correlate the above WWID to a Serial Number in the Storage Center GUI as below

In the highlighted Serial number above, the first part of the WWID is Storage Center’s unique ID, the middle part is made up of the controller number in hex, and the last part is the serial number of the volume. To ensure correct correlation in environments with multiple Dell Compellent Storage Centers, be sure to check the controller number as well.

The only situation where the two numbers would not correlate is if a Copy Migrate had been performed. In this case, a new serial number is assigned on the Storage Center side, but the old WWID must still be presented to the server so that the path to the server is not disrupted.

Configuring the discovered Compellent volumes using device-mapper

Device Mapper is a generic framework for block devices in Linux Operating System which provides single block device node for a multipathed Logical Unit and ensures I/O routing to available paths during path failure event.

Dell Compellent Storage Center supports the native Linux device-mapper utility to configure and manage the multipath Compellent storage volumes. The following steps provide the recommended method to configure the Compellent storage multipath volumes:

1. Once the disks are detected, the Device Mapper needs to be configured to allow multipathing for that particular volume.

Verify that your device-mapper and multipath driver are at least the version shown below or higher by running the command
 

# rpm -qa | grep device-mapper device-mapper-1.02.77-9.el6.x86_64 device-mapper-multipath-libs-0.4.9-64.0.1.el6.x86_64 device-mapper-multipath-0.4.9-64.0.1.el6.x86_64

2. Identify the disks for multipathing e.g. /dev/sdd and /dev/sde and obtain the scsi_id of the disk.

# scsi_id -g -u /block/sdd 36000d310000065000000000000000050 # scsi_id -g -u /block/sdd 36000d310000065000000000000000051

3. Edit the /etc/multipath.conf file and update as follows:

1. Add compellent disks wwid information in the blacklist_exceptions, for e.g.
 

blacklist_exceptions { wwid "36000d310000065000000000000000050" wwid "36000d310000065000000000000000051" }

2. Create an alias in the multipaths section
 

multipaths { multipath { wwid alias alias_of_volume1 } multipath { wwid alias DATA } }

 

 4. Restart your multipath daemon service using the following command:

# service multipathd restart

5. Verify that your multipath volumes alias are displayed properly by running the following command:

# multipath –ll DATA (36000d31000ece2000000000000000050) dm-2 COMPELNT,Compellent Vol size=200G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active |- 7:0:0:1 sdb 8:16 active ready running |- 7:0:1:1 sdc 8:32 active ready running |- 8:0:0:1 sdd 8:32 active ready running |- 8:0:1:1 sde 8:32 active ready running

6. Start the multipath service upon boot using the command:

# chkconfig multipathd on

Recommended HBA settings:

The multipathing functionality also depends on the port connectivity timeout of the HBA modules. This section describes the recommended settings related to a Qlogic HBA.

The port connectivity timeout settings determine how long the system waits to destroy a connection after losing connectivity at the port level. During a controller failover, the World Wide Name (WWN) for the active port will disappear from the fabric momentarily before returning on the reserve port on the other controller. This process can take anywhere from 5 to 60 seconds to fully propagate through a fabric. As a result, for a Qlogic HBA the default timeout of 30 seconds is insufficient and the value must be changed to 60. To view the current value:

QLogic:

# cat /sys/module/qla2xxx/parameters/qlport_down_retry 60

To setup the recommended timeout settings perform the following:

1. Create the qla2xxx.conf file

# vi /etc/modprobe.d/qla2xxx.conf

2. Add the below line in the qla2xxx.conf file:

options qla2xxx qlport_down_retry=60

3. In order for the new settings specified in a modprobe.d configuration file to be persistent across reboots, the boot ramdisk will need to be rebuilt using the following command:

# dracut -f –v

Reference:

-------------------------------------------------------------------------------------

• Whitepaper 'RHEL6_Best_Practices_with_Dell_Compellent_Storage_Center.pdf' from Dell Compellent available at Dell Compellent's Knowledge Center http://kc.compellent.com