In this discussion, I’d like to address the guidelines on network hops between EqualLogic arrays. The assumption is that the entire EqualLogic Group, comprised of pools, is deployed within a single Layer 2 Ethernet network, which is usually limited to connectivity within a data center. IP addresses for the arrays are within a single IP subnet and no Layer 3 routing is involved. In addition, physical layer interconnects based on Ethernet standards are used for interconnecting switches, arrays, and hosts (E.g. Cat 6 1GbE, 10GBASE-xx 10GbE technologies). The network is following best practices for a Layer 2 Ethernet network broadcast domain, such as span and scope.
Before deriving the recommendations, it will be useful to discuss the types of network traffic happening in the SAN:
Note: An EqualLogic array group comprises 1 to 4 pools. Each pool includes a sub-set of arrays in the group and an array belongs to only one pool at a time. Host volumes reside within a pool and are distributed across member arrays within the pool.
In designing a Layer 2 iSCSI SAN, network latency is an important criterion and is influenced by hop count. Hop count is the number of intermediate switches that a frame needs to traverse from a source to a destination device. The total network latency between devices increases with hop count. Network latency includes link delay (propagation delay on the medium) and internal switch delays. Switch delays include forwarding delays from input to output port; processing delays for things like security or QoS; and queuing/buffering delays due to congestion at an output port.
The inter-array traffic is sensitive to network latency as well as the host to array traffic. Sensitivity of inter-array traffic to latency depends on the type of traffic such as group management or data traffic. The high level guidelines can be split based on the traffic type as follows.
Keep the arrays within a pool connected across two switches (or two hops) at most and restrict the pool within a data center. Typically, traffic between arrays in a pool is due to latency sensitive activities such as load balancing between arrays and host data serving.
Keep all Group members connected by four switches (or four hops) at most. Typically, traffic between pools is due to group management communications and less sensitive to latency than arrays within a pool.
Below are three examples following the above recommendations. The above are just recommendations and obviously there can be exceptions based on various factors.
Pool connectivity = Maximum 2 hops; Group connectivity = Maximum 2 hops
The switch inter-connect can be a stack instead of a LAG.
Pool connectivity = Maximum 2 hops; Group connectivity = Maximum 4 hops
All switches are inter-connected via a LAG and the one denoted in dotted line is blocked by Spanning Tree Protocol (STP). The switch inter-connect can be a stack instead of a LAG.
A pair of stacked switches is connected via VLT to a pair of switches forming a VLT domain. A second pair is connected to the VLT domain in the same manner. Refer to this blog post for a discussion on switch inter-connects and VLT: http://en.community.dell.com/techcenter/b/techcenter/archive/2013/01/07/blade-server-topology-options-with-external-top-of-rack-switches-for-equallogic-arrays-part-1.aspx
Two or four switches in a flat model typically suffice for storage connectivity of an EqualLogic group (see example 1 or example 2). Connecting more than four switches requires the hierarchical model in example 3 because of the low port count on a selected switch model. Switch inter-connectivity design with four or more switches is complex when adhering to the hop recommendations.
Link to a follow up blog post on server network connectivity guidelines to EqualLogic SAN: http://en.community.dell.com/techcenter/b/techcenter/archive/2013/02/11/server-network-connectivity-guidelines-for-equallogic-san.aspx
Link to EqualLogic publications: http://en.community.dell.com/techcenter/storage/w/wiki/2631.storage-infrastructure-and-solutions-team.aspx