HPCatDell home button

Duke University: Higher Performance with Less Power - The Dell TechCenter


At Duke University, over the past two years, we've been collecting power consumption data across 500-600 machines of the Duke Shared Cluster Resource (DSCR). The DSCR is a large Linux cluster - over 4100 CPU-cores - that provides "production-grade" computational capabilities for over 70 research groups and 700 users from all across campus.

The data was presented at Dell’s booth at SC'10, and summary charts are available at: https://wiki.duke.edu/display/SCSC/Power+Usage+Data . I thought it was interesting to walk through recent history, with a focus on the power consumption of a cluster. The plots at the above URL all show CPU-load (0 to 100% usage of all CPU-cores in the machine) against power (Watts consumed by the server).

In 2008, we transitioned from 1U servers to blades, in part for the better power efficiency of the blade platform. Figure 2 shows a 100 to 150W drop in power for the full range of CPU load. While we attribute the majority of this decrease to the blade form-factor, note that we also transitioned from 1U to blade, and from an X5355 (high power draw) to E5420 (low power draw) CPU at the same time.

In 2009, we moved to the newer E5520 CPUs – Figure 3 – staying on the blade platform, although it is a newer-generation blade (Dell M610 vs M600). Across the board, we saw a 30W drop in power.

In late 2009 – Figure 4 – we had several professors purchase the higher-power X-series CPUs (note that the DSCR is a “condo model” where professors directly purchase hardware and contribute it to the cluster). These X-series CPUs do lead to increased power consumption, but it was interesting to see that they are relatively efficient at low-CPU, only 10-20W more power than the E-series for 0-20% load. At higher load, however, they do draw significantly more power – 40W more at 100% load.

In 2010 – Figure 5 – we transitioned to the X5650 CPUs and while the sample size for this data is rather small, the trend seems to suggest that the newer CPUs do not draw significantly more power than the X5550s, and they retain some efficiency at lower CPU-loading. Of course, the X5650s have 50% more CPU-cores and so watts-per-core (or watts-per-job) has decreased considerably.

As power and cooling become increasingly important to HPC installations, this data suggests that vendors such as Dell and Intel are responding in concrete and meaningful ways, with newer platforms that require less power while providing higher levels of performance.

---

Duke University: Higher Performance with Less Power - The Dell TechCenterJohn B. Pormann, Director of Research Computing, Director of Scalable Computing Support Center

Dr. John B. Pormann is the director for the Scalable Computing Support Center (SCSC). The SCSC is a service and support center managed by OIT that will connect Duke University researchers to hardware, software, educational and personnel resources, both local and global, to enable novel computational science. John worked in industry for several years, including a major DOD supercomputing site, and more recently with the CUDA software group at NVIDIA.

John has an undergraduate degree in Computer Engineering from Stevens Institute of Technology, and graduate degrees in Electrical and Computer Engineering from Duke University. He conducted post-doctoral research at Duke in cardiac electrophysiology, computational immunology and computational electromagnetics. His research interests are scientific computing, including application and training for high-performance and parallel computing techniques, as well as software engineering.