Chassis Management Controller Stops Functioning

1. Perform one of the following:

  • If you are able to access or ping the CMC IP address, log in to CMC though the Web interface or RACADM (Telnet/SSH, Remote, or Serial RACADM) interface.
  • If you log in to CMC through the Web interface, go to Chassis Overview → Troubleshooting → Reset components and click Reset/failover CMC.
  • If you log in to CMC through RACADM (Telnet/SSH/Serial), run the racadm cmcchangeover command. This command is applicable only if the M1000e chassis has two CMCs, one active and the other standby. If there is only one CMC in the M1000e chassis, run the racadm racreset command.

  • If you log in using remote RACADM and if the chassis has two CMCs (one active and the other standby), run the following command:

racadm -r<CMC IP address>-u<username>-p<password>cmcchangeover

It takes around five minutes for the CMC to reboot. Log in to CMC through the Web interface or RACADM (Telnet/SSH/Serial) using the IP address and view the active errors in the CMC logs.

  • CMC Web Interface: Go to Chassis Overview → Logs to view the CMC logs.
  • RACADM: Use the getsel, getraclog, or gettracelog commands.

2. If the problem persists or if you are unable to access the CMC IP, connect a Serial cable to the CMC. After the login prompt is displayed, check the network settings using the following command:

racadm getconfig -g cfglannetworking


To change the network settings such as IP address, Subnet address, or Gateway address from DHCP to static IP or from static IP to DHCP, run the following command:

racadm help config -g cfglannetworking

Follow the examples provided in the output.

3. If the problem persists or if you are unable to access CMC from Serial connection, then physically remove and reinsert CMC in the chassis.
During this operation, IP address of all servers and iDRAC is temporarily inaccessible. It takes around five minutes for the CMC to reboot. Log in to CMC through the Web interface or RACADM (Telnet/SSH/Serial) using the IP address and view the active errors in the CMC logs.

  • CMC Web Interface: Go to Chassis Overview → Logs to view the CMC logs.
  • RACADM: Use the getsel, getraclog or gettracelog commands.

4. Connect the network cable to the left most RJ45. If the problem persists, and you are able to access CMC through a Serial connection, then check for a recover prompt. If you see a recover prompt, type recover and then press <Enter>. One of the following messages is displayed:

recover1[self test] CMC1 self test failure

recover2[Bad FW images] CMC2 has corrupted images

If the prompt indicates a self test failure, there are no serviceable components on CMC. Then, replace the CMC card.
If the prompt indicates Bad Firmware Images, then recover the firmware image:

a. At the recover prompt, type recover and then press <Enter>. The recover reason and available sub-commands are displayed.
b. Run the following commands to reprogram the flash devices by uploading the firmware update file firmimg.cmc:
recover getniccfg

recover setniccfg <IP address> <Subnet address> <Gateway address>

recover ping <IP address>

recover fwupdate -g -a <IP address>

NOTE: This is the same firmware image file that is used for normal firmware updates.

NOTE: In recover mode, you cannot ping CMC normally because there is no active network stack. The recover ping <TFTP server IP> command allows you to ping the TFTP server to verify the LAN connection. You may need to use the recover reset command after setniccfg on some systems.


The recovery process displays its current activity and boots to the CMC OS upon completion. An example recover sequence may be:
recover getniccfg

recover setniccfg 192.168.0.120 255.255.255.0 192.168.0.1

recover ping 192.168.0.100

recover fwupdate -g -a 192.168.0.100

NOTE: Make sure CMC has the latest revision firmware. To check the CMC version, from Telnet/SSH use the racadm getversion command or on the CMC Web interface, go to Chassis Overview → Update.
To get the latest CMC firmware visit the Dell Support site at Support → Drivers & Downloads → Enterprise IT → Servers, Storage & Networking → Poweredge → Poweredge M1000E → Chassis System Management → Dell firmware.

***************************************************************************************************************

Power Supply Unit Not Detected

1. Make sure that all the Power Supply Units (PSUs) are firmly connected to the chassis and status for each PSU is either online or standby. Check and replace the AC or DC cord if needed. Check to confirm the power distribution unit providing power to the PSU is operating normally.

2. Make sure that all the PSUs are operating at the same voltage. Use external tools to check and correct the voltages for the PSUs.

3.If the problem persists, perform one of the following:

  • If you are able to access or ping the CMC IP address, log in to CMC though the Web interface or RACADM (Telnet/SSH, Remote, or Serial RACADM) interface.
  • If you log in to CMC through the Web interface, go to Chassis Overview → Troubleshooting → Reset components and click Reset/failover CMC.
  • If you log in to CMC through RACADM (Telnet/SSH/Serial), run the racadm cmcchangeover command. This command is applicable only if the M1000e chassis has two CMCs, one active and the other standby. If there is only one CMC in the M1000e chassis, run the racadm racreset command.
  • If you log in using remote RACADM and if the chassis has two CMCs (one active and the other standby), run the following command:

racadm -r<CMC IP address>-u<username>-p<password>cmcchangeover

It takes around five minutes for the CMC to reboot. Log in to CMC through the Web interface or RACADM (Telnet/SSH/Serial) using the IP address and view the active errors in the CMC logs.

* CMC Web Interface: Go to Chassis Overview → Logs to view the CMC logs.
* RACADM: Use the getsel, getraclog or gettracelog commands.

Check if the PSU comes online after this action. To verify, in the CMC Web interface, go to Power Supplies → Properties. The Power Supply Status page is displayed. The Power Status value should be Online for each PSU. If the problem persists, contact Dell Technical Support to replace the PSU.


******************************************************************************************************************************

Modular Server in the Chassis Does Not Power On

1. Make sure that all the Power Supply Units (PSUs) are firmly connected to AC or DC cord and status for each PSU should be online or standby. Check the system input power cap setting in the Budget/Redundancy Configuration section of the CMC Web interface (Chassis Overview → Power → Configuration .) If the power cap setting value is configured too low, then it does not allow any additional servers to turn on.

2. If the problem persists, check for 110V operation. If any PSU is connected to a 110V branch circuits, verify if this is a valid configuration before servers are allowed to turn on. For more details, see the Power Configuration settings in the Budget/Redundancy Configuration section of the CMC Web interface.
Check if the PSU status changes to online after this action. If the problem persists, contact the Dell Technical Support to replace the PSU.

3. If the problem persists, select the Max Power Conservation setting in the Budget/Redundancy Configuration section of the CMC Web interface.
In the Maximum Power Conservation mode, all servers function at their minimum power threshold, and all subsequent server power allocation requests are denied. In this mode, the performance of the powered on servers may be reduced. Additional servers cannot be powered on, regardless of server priority. The system is restored to full performance when the Maximum Power Conservation option is cleared. For more details, see the Power Configuration settings in the Dell Chassis Management Controller Firmware  User’s Guide available at dell.com/support/manuals.

4. If the problem persists, check the server slot power priority of the slot associated with the newly inserted server, and make sure it is not lower than any other server slot power priority.
The least priority servers lose power after a PSU failure. This is the expected behavior if the enclosure power policy was configured to No Redundancy.

5. To avoid a future power supply failure causing servers to turn off, make sure that the chassis has at least four PSUs. To prevent PSU failure from impacting server operation, make sure that the chassis is configured for the Power Supply Redundancy policy.