Applies to:

 Storage Platform(s) – Dell Compellent Series 40 storage systems with Storage Center 5.4
 and greater

Author:

Wendy Chen

Problem:

How do I cable my Compellent array for Oracle RAC?



Solution:

 Series 40 Controller Overview

 The Dell Compellent Series 40 controller is released with Storage Center 5.4.  The following diagram shows the back view of the Series 40 controller. 

 

The Dell Compellent Storage Center controllers are used to support various I/O adapters including Fibre Channel, iSCSI, FCoE, and SAS. 

As shown in the controller back view, Series 40 controllers have 7 PCI-E I/O slots.  Slot 7 is populated with the cache card.  I/O cards with front end ports are installed from slot 6 and fill toward 1.  I/O cards with back end ports are installed from slot 1 and fill toward 6.

The Eth0 LAN port is for the management traffic.  The Eth1 LAN port is used for the heart beat connection between controllers in a multi-controller system.

Series 40 Controller Back End Cabling to Enclosures

 The Dell Compellent Series 40 storage controllers support both Fibre Channel and SAS (Serial Attached SCSI) for the back-end architecture to the storage enclosures depending on the enclosure type.  The Series 40 controllers support the following enclosures that are available in three different technology types:

    1. SAS – Serial-Attached SCSI (available with Storage Center 5.0 and greater).  Supported communication speed between controllers and SAS enclosures is 3 Gbps or 6 Gbps per channel with each cable containing 4 channels.
    2. Fibre Channel – Switched Bunch of Disks (SBOD).  Supported communication speed between controllers and SBOD is 2 Gbps or 4 Gbps.
    3. SATA – Serial Advanced Technology Attachment

The following diagram illustrates the back end cabling details of the dual Series 40 controllers with two SBOD enclosures and two 6 Gb SAS enclosures.  Dual controllers are recommended in order to meet the high availability requirement of Oracle databases.  Cabling should follow Dell Compellent recommended best practices to provide redundancy and high availability. 

On the SBOD enclosures, all inward connections are connected to the Host 0 port and all outward connections are connected to the Host 1 port.  Connect A sides of all SBOD enclosures together with one loop; connect B sides together with another loop.  The cabling of the SBOD enclosures in the above diagram depicts the followings:

 Loop 1, Path 1 (as shown in the blue cabling):

  Controller 1 to Enclosure 1 to Enclosure 2 to Enclosure 3 to Controller 2

 Loop 1, Path 2 (as shown in the red cabling):

  Controller 2 to Enclosure 1 to Enclosure 2 to Enclosure 3 to Controller 1

 On the 6 Gb SAS enclosures (2.5’’ enclosures or 3.5’’ enclosures), the port labeled C connects to a 6 Gb HBA on the controller.  Connect top I/O modules of all SAS enclosures together with one chain; connect bottom I/O modules together with another chain.  All open Controller SAS ports must be cross connected to opposite ports on the second Controller.  This provides for the Inter Processing Communication (IPC).  The cabling of 6 Gb SAS enclosures in the above diagram depicts the followings:

 Chain 1, Side 1 (as shown in the green cabling):

Controller 1 Initiator only port 1 -> C port on Enclosure 1 top IO module

B port on Enclosure 1 top IO module -> A port on Enclosure 2 top IO module

C port on Enclosure 2 top IO module -> Initiator / Target port 3 on Controller 2

 
Chain 1, Side 2 (as shown in the purple cabling):

Controller 2 Initiator only port 1 -> C port on Enclosure 1 bottom IO module

A port on Enclosure 1 bottom IO module -> B port on Enclosure 2 bottom IO module

C port on Enclosure 2 bottom IO module -> Initiator / Target port 3 on Controller 1

 
Crossover 1, Side 1 (as shown in the orange cabling):

Controller 1 Initiator only port 2 -> Controller 2 Initiator / Target port 4

 
Crossover 1, Side 2 (as shown in the dark green cabling):

Controller 1 Initiator / Target port 4 -> Controller 2 Initiator only port 2

 For more details on the back end cabling, refer to the Dell Compellent Storage Center 5.4 System Connectivity Guide, and the Dell Compellent Storage Center 5.4 SAS User Guide.

Series 40 Controller Front End Cabling to Host Servers

 The Series 40 controllers support the following protocols for the front end cabling to transmit data between host servers and controllers:

  1. Fibre Channel - Supported communication speed between controllers and host servers is 2 Gbps, 4 Gbps or 8 Gbps.
  2. iSCSI – Supported speed is 1 Gbps or 10 Gbps.
  3. FCoE – supported speed is 10 Gbps.

 In order to maintain continuous connectivity to stored data from the server, fault domains are established to create redundant I/O paths.  These fault domains provide for continuous connectivity with no single point of failure and without loss of bandwidth.  Fault domains provide fault tolerance at the controller level. 

 With Storage Center version 5 and above, the new feature virtual ports can be used to implement fault domain.  In addition to controller level redundancy, virtual ports can also offer port level redundancy.  At a minimum, Compellent requires two ports in the same fault domain on the same controller to support port level failover.

 The following diagram demonstrates an example of front end cabling of a two node clustered servers to the dual controllers through the dual Fibre Channel switches.  Domain 1 (as shown in the yellow solid and dot cablings) includes all connections to the left switch with two connections from each of the two controllers.  Domain 2 (as shown in the blue solid and dot cablings) includes all connections to the right switch with two connections from each of the two controllers.  In this example, if a physical port fails, the virtual port will move to another physical port within the same fault domain and on the same controller. 

 

References

 Dell Compellent Storage Center 5.4 System Connectivity Guide

 Dell Compellent Storage Center 5.4 SAS User Guide