The latest Dell NSS-HA solution was published on October 2012, of which the version is NSS4.5-HA. This release leverages the latest Dell PowerVault Storage stack (MD3260 and MD3060e) to offer denser storage solutions than previous NSS-HA solutions (NSS2-HA, NSS3-HA, and NSS4-HA).

Furthermore, as an extension of NSS4.5-HA configuration, the NSS4.5-HA XL configuration was developed. Figure 1 and Figure 2 show the design of NSS4.5-HA and NSS4.5-HA XL configurations, respectively. The major difference between NSS4.5-HA and NSS4.5-HA XL configurations is that NSS4.5-HA is only able to support one MD3260 + MD3060E storage stack, while NSS4.5-HA XL is able to support two MD3260 + MD3060E storage stacks concurrently. Thus, for the sake of simplicity, NSS4.5-HA XL configuration can be considered as two NSS45-HA configurations, except the two configurations share the same pair of PowerEdge R620 servers.

Figure 1. NSS4.5-HA 360TB architecture

Figure 2. NSS4.5-HA XL 2x360TB architecture

It is also worth mentioning that there are two pairs of active-passive servers configured for two storage stacks in XL configuration, each pair only hosts the I/O requests for one storage stack. For example, two PowerEdge R620 servers are labeled by “active” and “passive”, respectively.

  • Pair 1: server “active” handles I/O requests to/from storage stack 1, server “passive” is standby for the storage service running at “active”.
  • Pair 2: server “passive” hosts storage stack 2, server “active is standby for the storage service running at “passive”.

Thus, once server “active” suffers a catastrophic failure, the storage service hosted by “active” will automatically fail over to server “passive”; similarly, once “passive” fails, the storage service running at it will fail over to “active”.

For detailed information about the XL configuration, please refer to Dell NFS Storage Solution with High Availability – XL configuration.  

 

Although Dell NSS-HA solutions have received many hardware and software upgrades to support higher availability, higher performance, and larger storage capacity since the first NSS-HA release, the design architecture and guideline of the NSS-HA solution family remain unchanged. Thus, for the rest of the blog, only the deployment and sequential I/O performance information of NSS4.5-HA and NSS4.5-HA XL will be presented.

For detailed information about NSS-HA solutions, please refer to our published white papers:

 

NSS4.5-HA and NSS4.5-HA XL deployment summary

Table 1 shows the six items for successfully deploying NSS4.5-HA and NSS4.5-HA XL configurations:

  • Usage: specifies the hardware setup options for an NSS-HA configuration.
  • Configuration guide: specifies the name of the document for deploying an NSS-HA configuration.
  • HA cluster configuration utility: specifies the name of the utility for configuring HA cluster. It is used to generate HA cluster configuration file.
  • File system configuration utility: specifies the name of the utility for file system configuration. It is used to configure storage devices, including creating/removing physical volumes, volume groups, logical volumes, and the XFS file system.
  • PowerVault storage configuration utility: specifies the name of a utility for PowerVault storage configuration. It is used to configure the PowerVault MD3260 and MD3060e storage stacks.
  • HA cluster monitor utilities: specifies the names of utilities for monitoring HA cluster components.
    • sas_path_check.sh: It is used by Red Hat HA cluster management tool to monitor the status of SAS paths.
    •  ibstat_script.sh: It is used by Red Hat HA cluster management tool to monitor the status of IB links, if IPoIB is deployed.

 

Table 1 NSS4.5-HA vs. NSS4.5-HA XL

 

NSS4.5-HA configuration

NSS4.5-HA XL configuration

Usage

Only two options:

  • 180TB raw storage capacity.
  • 360TB raw storage capacity.

Only one options:

  • 2 x 360TB raw storage capacity (two independent file systems).

 

Configuration guide

NSS4.5_HA_recipe_v1.0.pdf

NSS4.5_HA_XL_recipe_v1.0.pdf

 

HA cluster configuration utility

cluster_config.sh

cluster_config_xl.sh

 

File system configuration utility

nssha63_single.py

 

nssha63_xl.py

PowerVault storage configuration utility

  • 180TB: MD3260_180TB.scr
  • 360TB: MD3260_360TB.scr

 

There are two utilities for XL configuration:

  • MD3260_360TB_1.scr
  • MD3260_360TB_2.scr

 

HA cluster monitor utilities

sas_path_check.sh

 

ibstat_script.sh

 

Please refer to the attachments of the blog for all configuration guides and utilities mentioned above.

Note: for any customized configuration/deployment, please contact your Dell representative for specific guidelines.

NSS4.5-HA and NSS4.5-HA XL sequential I/O performance summary

For NSS45-HA XL configuration, there are two scenarios when clients are accessing two storage stacks concurrently.

  • Failure-free: there is no failure. Each PowerEdge R620 server handles I/O requests with only one storage stack (MD3260 + MD3060E).
  • Failure: there is a failure occurred in a PowerEdge R620. Thus, the other server handles I/O requests with two storage stacks concurrently, instead of one, as the storage service from the failed server will be transferred (fail over) to the healthy one.  

It is worth discussing the I/O performance behaviors between the two cases. Figure 3 and Figure 4 present the sequential write and read performance numbers collected from NSS4.5-HA configuration, and two scenarios of NSS4.5-HA XL configuration.

  • “NSS45-HA” denotes the performance numbers for NSS4.5-HA standard configuration.
  • “NSS45-HA-XL-single-server” denotes the performance numbers collected in the failure scenario for NSS4.5-HA XL configuration.
  • “NSS45-HA-XL-two-servers” denotes the performance numbers collected in the failure-free scenario for NSS4.5-HA XL configuration.

Note: the performance benchmarking methodology are same when collecting the three sets of the performance numbers, except that for XL configuration, half of clients are accessing one storage stack, the other half are accessing the other storage stack, thus, the performance of XL configuration is the overall performance of the two storage stacks.  For detailed benchmarking information, please refer to NSS4.5-HA white paper, section 5, page 17.  

Write performance

As mentioned above, the write performance of XL configuration is the aggregate write performance for accessing the two storage stacks concurrently. For example, in a 32-client test case, all 32 client nodes access a single storage stack for NSS4.5-HA standard configuration, while for XL configuration in any scenario; each storage stack only handles the concurrent I/O requests from 16 client nodes. Thus, by processing the I/O requests in two different storage stacks concurrently instead of a single storage stack, it is expected to observe that the overall write performance numbers of XL configuration in any scenario are general twice better than the ones of the NSS4.5-HA standard configuration. The results presented in Figure 3 confirm the expectation.

It is also worth pointing out that there is little write performance difference between the two scenarios for NSS4.5-HA XL configuration, as shown in Figure 3. It is reasonable, because for the sequential write workloads, the storage stack itself is the bottleneck for the entire solution, thus, it helps little for write performance by increasing the computation power and network bandwidth (adding one more R620).

Figure 3. Sequential write performance

Read performance

As shown in Figure 4, it is interesting to point out that the read performance of XL configuration in a failure scenario is almost half of the one in a failure-free scenario. Should it be expected that XL configuration always has similar read performance in any scenario as its write performance? The answer is NO. As discussed above, due to the slow write request processing speed of a storage array, write performance of XL configuration in the two scenarios is similar, which is independent of computation power and network bandwidth. While, the PowerVault MD3260 and MD3060E storage array has much faster read request processing speed than write request processing speed; 4 GB/sec can be achieved according to our server-to-storage test records. Thus, in a failure scenario, the computation power and network bandwidth in a single R620 become insufficient to handle high volume of read requests for the two storage array concurrently. As shown in Figure 4, such resource insufficiency not only makes the read performance of XL configuration twice worse than the one of XL configuration in a failure-free scenario, but also makes it worse than the one of NSS4.5-HA standard configuration with the increase of concurrent read requests. It is also worth mentioning that once the system is recovered from the failure scenario, the I/O workload will be balanced again, and a big overall performance improvement will be observed.

Figure 4. Sequential read performance