Best Practices in Power Savings on Dell PowerEdge Servers by MR, Rajkumar (01-27-2011)

Introduction:
Energy efficiency is currently one of the most important topics in the computing industry, especially in the server space. To reduce power usage, we need to implement an energy-efficient server infrastructure that facilitates power management and that helps allocate power to computing resources as needed. Power savings are also made possible by virtualization technology. To take full advantage of virtualization technology and other resources, we need to make sure the hardware that we select allows for maximum energy efficiency. In some places where we can’t sacrifice performance, we need to explore alternatives. In this paper I have listed best practices for reducing power consumption effectively.

Memory
Memory module power consumption varies widely from one module to another. Bus speed plays a large factor in memory power consumption, but so do density, rank, and operating voltage. On a system that has many sticks of RAM, the memory power footprint becomes a large percentage of system power and a prime target for savings. When considering memory options, the five key considerations are price, performance, power, RAS features, and scalability.

A wide variety of memory options are available. These include Unregistered DIMMs (UDIMMs) and Registered DIMMs (RDIMMs) at various speeds and capacities. RDIMMs should be purchased by customers who need large amounts of memory (up to 8 GB DIMMs). UDIMMs should be purchased by customers who only require a limited amount of memory and are looking for power and cost savings. RDIMMs use about 1 W of power more per DIMM than comparable UDIMMs due to the register feature. However, RDIMMs will allow for performance improvements when the memory is being highly utilized and more than 1 DIMM per channel is populated. Both UDIMMS and RDIMMS use less power than DDR2 FBD DIMMs.

A memory rank is an area or block of 64-bits (72 bits for ECC memory) created by using some or all of the DRAM chips on a DIMM. Because of larger capacity, single-rank DIMMs are manufactured with higher-density DRAM chips and they typically cost more than dual-rank DIMMs of comparable capacity. If your hardware supports them, instead of selecting 4 sticks of 4GB DIMM, you can choose 2 sticks of 8GB DIMM .Of course, you should evaluate if this choice achieves your business requirements.


Storage
You can achieve significant power reductions by selecting power-efficient hard disks for your server. Even if the absolute power savings per disk may seem small, quantities of hard disks that are deployed in a data center can easily number in the thousands. At this scale, choosing power-efficient hard disks can save many kilowatts of power.

Size Reduction
One way to save power associated with storage is to replace 3.5-inch with 2.5-inch disk drives. This is one of the few scenarios in which saving power does not compromise performance, and price differences can be reduced by the long-term cost savings associated with the new platform. The reduction in platter size and weight means that the disk can use smaller actuators and more power-efficient motors, which results in a large power savings. In addition, the performance of 2.5-inch drives is generally equal to or better than that of 3.5-inch drives, with a few caveats for specific workloads and data layouts. The complicating factor is that the sheer area of a 3.5-inch drive allows for higher capacity per spindle, which 2.5-inch drives have not yet achieved. Currently, it may be unfeasibly expensive or impossible to use 2.5-inch drives to build low-power, high-capacity installations.

RPM Reduction
Enterprise-class, 15,000-RPM disks offer the highest performance from rotational media disk drives, but they are not always necessary. In many scenarios, you can deploy lower RPM disk drives (such as 10,000 or even 7,200 RPM) in a storage array and still accomplish your business goals.

Solid-State Disks
Solid state disks (SSDs) have no magnetic platters to rotate, eliminating power costs for disk motors and actuators—which are the most significant items on the power budget for rotational drives. Although client rotational drives spin their media down at idle periods to save power, enterprise drives do not. Therefore, SSDs generally use much less power than hard disk drives (HDDs). However, SSD technology is changing rapidly. If you are considering SSDs for your IT environment, you must consider not only their power usage, but also their expected lifetime and how much your application workload can take advantage of solid-state storage technology.

Moving to a higher storage capacity
Hard disk size plays a role in power management i.e. instead of selecting two 500GB HDD, select a single 1 TB HDD. When creating RAID 5 configuration, instead of having six 500 GB HDDs, we can have three 1 TB HDDs, which will save a lot of power. Of course, you should evaluate if the performance, redundancy, and availability of HDD can still achieve your business requirements.

Hard Disk Selection
Based on the requirements for storage capacity in your local ecosystem, you can select the server model that best fits your needs. For example, a Dell PowerEdge R510 provides numerous options for drive capacity (customers can choose 4, 8, or 12 drives).

RAID Selection
In the past, power has not generally been a factor in choosing a RAID setup. However, power capacity can influence this decision. If your infrastructure is approaching its maximum power capacity, the power budget of large RAID arrays can make expansion of existing RAID setups or the purchase of new setups impossible. If replacing existing 3.5-inch disks with 2.5-inch disks of identical capacity is not possible and your organization is at its power capacity limit, one way to save some power is to change your RAID setup. For example, you might move from a higher degree of reliability and performance (RAID 10) to a lower degree (RAID 5). Of course, you should evaluate if the performance, redundancy, and availability of different RAID setups can still achieve your business requirements.

Network Adapters
To minimize network adapter power consumption, purchase only as much capacity as you need. If a server has low utilization or does not require a large amount of bandwidth, purchasing the highest throughput network adapters is unnecessary and consumes more power. For example, typical power consumption for a 1‑GBPS PCIe network adapter is less than a 10‑GBps PCIe card. Another thing to consider is the number of ports on the network card. If four connections are required, a single four-port card consumes significantly less power than four individual single-port cards. As an example, current quad-port 1‑GBps cards consume less power than four 1‑GBps single-port cards.

Storage Adapters
FCoE (Fibre Channel over Ethernet) plays a major role in today’s Tech industry. Because FCoE adapters run both IP and storage traffic, the potential exists to replace four standalone storage and network adapters with two FCoE converged network adapters, thus reducing power consumption. Even though Fibre Channel over Ethernet is a costlier solution compared to iSCSI, it will be helpful in future to consumers.

Power-Efficient Processors
Power management features have been built into processors for several years, and processor manufactures are working on additional power management features for future product lines. Some processor families incorporate low-power states, whereas other processor families are designed to take advantage of low-voltage parts. Either type can save significant wattage.

Turbo Mode and C-States
Turbo mode allows processor cores to run faster than their base operating frequency. To take advantage of Turbo mode, enable Turbo mode in BIOS. To maximize the chances of entering a Turbo mode, C-states should be enabled as well. The BIOS power management profile of Dell servers can be set to any of these available options. The C1E (Enhanced halt state) is a new BIOS option which is added to Dell server from 11g onwards.

Power savings are achieved by reducing the frequency multiplier (Frequency identifier or FID) and the voltage (Voltage identifier or VID) of the CPU. Intel’s version of this technology is known as either Enhanced Intel SpeedStep Technology (EIST) or Demand Based Switching (DBS) and AMD’s version is marketed under the name Cool’n’Quiet or PowerNOW! The combination of a specific CPU frequency and voltage is known as a performance state (p-state). To allow the operating system to save the maximum amount of power at a wide range of frequencies, the ACPI specification has flexibility in how P-States are implemented. There are different P-states available from P0, P1, and Pn.

BIOS based P-state management
A grouping of a set of different power parameters is called a Power profile. From 11th Generation Dell servers onwards, the Dell PowerEdge BIOS setup provides various options for the power profile:

OS Control, Active Power Controller, Static Max Performance, and Custom.
Active Power Controller mode is the default mode. Dell 11G servers also introduce BIOS‐level, demand‐based power management (DBPM) profiles beyond those specified by the microarchitecture. Using a custom power profile, parameters like CPU Power and performance management, Fan control Algorithm and Memory Power and performance Management can be set. The table below shows us the settings of each power profile.



Static MAX Performance DBPM Disabled ( BIOS will set P-State to MAX) Memory frequency = Maximum Performance Fan algorithm = Performance
OS Control Enable OS DBPM Control (BIOS will expose all possible P states to OS) Memory frequency = Maximum Performance Fan algorithm = Power
Active Power Controller Enable DellSystem DBPM (BIOS will not make all P states available to OS) Memory frequency = Maximum Performance Fan algorithm = Power
Custom CPU Power and Performance Management: Maximum Performance | Minimum Power | OS DBPM | System DBPM Memory Power and Performance Management: Maximum Performance |1333Mhz |1067Mhz |800Mhz| Minimum Power Fan Algorithm Performance | Power



To provide customer choice and improve power efficiency, Dell supports both OS Power management along with Dell Active Power controller (DAPC), The figure below shows how change the power profile on Dell BIOS to take advantage of DAPC.


Dell BIOS power management settings


CPU Power and Performance Management Settings

CPU Power and Performance Management Settings


Conclusion
Using the methods outlined in this paper can lead to significant power savings. Even though some improvements may initially yield only incremental power savings, small changes add up when you consider power saved over the entire life of a server. Rising energy costs and the need to maximize energy-efficiency have become critical success factors for global IT operations, so make sure you intelligently select hardware and optimize settings to save power.


Reference
http://www.dell.com/downloads/global/products/pedge/en/server-poweredge-11g-white-paper.pdf
http://en.community.dell.com/dell-blogs/enterprise/b/tech-center/archive/2009/04/08/nehalem-and-memory-configurations.aspx
http://www.microsoft.com/whdc/system/pnppwr/powermgmt/Svr_Pwr_ITAdmin.mspx