How to solve the problem MD3000i WARNING: Restart by watchdog time out - DELL|EMC Storage Forum - Storage - Dell Community

How to solve the problem MD3000i WARNING: Restart by watchdog time out

Storage

Storage
Information and ideas on Dell storage solutions, including DAS, NAS, SAN and backup.

How to solve the problem MD3000i WARNING: Restart by watchdog time out

This question is not answered

Adding 9767 symbols for standalone.
Error
07/30/13-12:37:53 (GMT) (tRootTask): NOTE:  I2C transaction returned 0x0423fe00
 
Reset, Power-Up Diagnostics - Loop 1 of 13600 Processor DRAM
     01 Data lines                                                  Passed
     02 Address lines                                               Passed
3300 NVSRAM
     01 Data lines                                                  Passed
5900 Ethernet 91c111 #1
     01 Register read                                               Passed
     02 Register test                                               Passed
3A00 NAND Flash
     06 Bad Blocks Test                                             Passed
2310 Application Accelerator Unit
     01 AAU Register Test                                           Passed
6D00 LSI SAS 1068 IOC--Base Board
     01 IOC Register Read Test                                      Passed
     02 IOC Register Address Lines Test                             Passed
     03 IOC Register Data Lines Test                                Passed
6F01 QLOGIC EP4032 CHIP 0
     01 Register Read Test                                          Passed
     02 Register Address Lines Test                                 Passed
     03 Register Data Lines Test                                    Passed
3900 Real-Time Clock
     01 RT Clock Tick                                               Passed
Diagnostic Manager exited normally.
 
Current date: 07/30/13  time: 05:38:59
 
Send <BREAK> for Service Interface or baud rate change
07/30/13-12:38:12 (GMT) (tRAID): NOTE:  Set Powerup State
07/30/13-12:38:12 (GMT) (tRAID): NOTE:  SOD Sequence is Normal, 0
07/30/13-12:38:12 (GMT) (tRAID): NOTE:  SOD: removed SAS host from index 0
07/30/13-12:38:12 (GMT) (tRAID): NOTE:  In iscsiIOQLIscsiInitDq.  iscsiIoFstrBase = 0x0
07/30/13-12:38:12 (GMT) (tRAID): NOTE:  Turning on tray summary fault LED
07/30/13-12:38:14 (GMT) (tRAID): NOTE:  SYMBOL: SYMbolAPI registered.esmc0: LinkUp event
07/30/13-12:38:15 (GMT) (tNetCfgInit): NOTE:  Network Ready
07/30/13-12:38:17 (GMT) (tRAID): NOTE:  Initiating Drive channel: ioc:0 bringup
07/30/13-12:38:20 (GMT) (tRAID): NOTE:  IOC Firmware Version: 00-24-62-00
07/30/13-12:38:29 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:1 phy:0 prevNumActivePhys:2 numActivePhys:2
07/30/13-12:38:29 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:1 phy:1 prevNumActivePhys:2 numActivePhys:2
07/30/13-12:38:29 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:0 phy:2 prevNumActivePhys:2 numActivePhys:2
07/30/13-12:38:30 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:0 phy:3 prevNumActivePhys:2 numActivePhys:2
07/30/13-12:38:30 (GMT) (tSasCfg004): NOTE:  Alt Controller path up - chan:1 phy:18 itn:1
07/30/13-12:38:30 (GMT) (tSasCfg006): NOTE:  Alt Controller path up - chan:0 phy:16 itn:2
07/30/13-12:38:30 (GMT) (tRAID): NOTE:  IonMgr: Drive Interface Enabled
07/30/13-12:38:31 (GMT) (tRAID): NOTE:  SOD: Instantiation Phase Complete
07/30/13-12:38:31 (GMT) (tRAID): NOTE:  Inter-Controller Communication ChannelsOpened
07/30/13-12:38:31 (GMT) (IOSched): NOTE:  New Initiator:  1 - channel: 1,devHandle: x0b, SAS Address: 50024e847f724f01
07/30/13-12:38:31 (GMT) (IOSched): NOTE:  New Initiator:  2 - channel: 0,devHandle: x10, SAS Address: 50024e847f724f00
07/30/13-12:38:31 (GMT) (tSasDiscCom): NOTE:  SAS Discovery complete task spawned
07/30/13-12:38:31 (GMT) (tRAID): NOTE:  LockMgr Role is Slave
07/30/13-12:38:31 (GMT) (sasCheckExpanderSet): NOTE:  Expander Firmware Version: 0116-e05c
07/30/13-12:38:31 (GMT) (sasCheckExpanderSet): NOTE:  Expander SAS address: Hi = x50024e84 Low = x7f721810
07/30/13-12:38:33 (GMT) (tRAID): NOTE:  spmEarlyData: Using cached data
07/30/13-12:38:43 (GMT) (tSasDiscCom): WARN:  SAS: Initial Discovery Complete Time: 30 seconds
07/30/13-12:38:43 (GMT) (tRAID): NOTE:  WWN baseName 000e0024-e87f724f (valid==>AltMacMatch)
07/30/13-12:38:43 (GMT) (tRAID): NOTE:  IonMgr: Host Interface Enabled
07/30/13-12:38:43 (GMT) (tRAID): NOTE:  SOD: Pre-Initialization Phase Complete
07/30/13-12:38:53 (GMT) (tRAID): NOTE:  ACS: autoCodeSync(): Process start. Comm Mode: 0, Status: 1
07/30/13-12:38:53 (GMT) (tRAID): NOTE:  SOD: Code Synchronization Initialization Phase Complete
07/30/13-12:38:53 (GMT) (NvpsPersistentSyncM): NOTE:  NVSRAM Persistent Storageupdated successfully
07/30/13-12:38:53 (GMT) (tRAID): NOTE:  USM Mgr initialization complete with 0 records.
07/30/13-12:38:53 (GMT) (tRAID): NOTE:  EDR - recieved 1 small records
07/30/13-12:38:53 (GMT) (tRAID): NOTE:  EDR - recieved 0 large records
07/30/13-12:38:54 (GMT) (tRAID): NOTE:  Acquire              0.017 secs
07/30/13-12:38:55 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0
07/30/13-12:39:22 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion timeout
07/30/13-12:39:22 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion timeout, cmd = 0x69
07/30/13-12:39:23 (GMT) (tRAID): NOTE:  Qlogic coredump file written to '390VF4J:/tmp/QLogic_Coredump_port_0_390VF4J',rc 204E50, expected 204E50
07/30/13-12:39:23 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose returned -1
 
07/30/13-12:39:23 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
07/30/13-12:39:23 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed.  Stat f000
07/30/13-12:39:23 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State 4543
07/30/13-12:39:23 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
07/30/13-12:39:24 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0
07/30/13-12:39:51 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion timeout
07/30/13-12:39:51 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion timeout, cmd = 0x69
07/30/13-12:39:52 (GMT) (tRAID): NOTE:  Qlogic coredump file written to '390VF4J:/tmp/QLogic_Coredump_port_0_390VF4J',rc 204E50, expected 204E50
07/30/13-12:39:52 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose returned -1
 
07/30/13-12:39:52 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
07/30/13-12:39:52 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed.  Stat f000
07/30/13-12:39:52 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State 4543
07/30/13-12:39:52 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
07/30/13-12:39:53 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0
07/30/13-12:40:20 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion timeout
07/30/13-12:40:20 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion timeout, cmd = 0x69
07/30/13-12:40:21 (GMT) (tRAID): NOTE:  Qlogic coredump file written to '390VF4J
:/tmp/QLogic_Coredump_port_0_390VF4J',rc 204E50, expected 204E50
07/30/13-12:40:21 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose returned -1
 
07/30/13-12:40:21 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC07/30/13-12:40:21 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
07/30/13-12:40:21 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State 4543
07/30/13-12:40:21 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
07/30/13-12:40:22 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0
07/30/13-12:40:50 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion timeout
07/30/13-12:40:50 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion timeout, cmd = 0x69
07/30/13-12:40:50 (GMT) (tRAID): NOTE:  Qlogic coredump file written to '390VF4J:/tmp/QLogic_Coredump_port_0_390VF4J',rc 204E50, expected 204E50
07/30/13-12:40:50 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose returned -1
 
07/30/13-12:40:50 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
07/30/13-12:40:50 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed.  Stat f000
07/30/13-12:40:50 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State 4543
07/30/13-12:40:50 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
07/30/13-12:40:52 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0
07/30/13-12:41:19 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion timeout
07/30/13-12:41:19 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion timeout, cmd = 0x69
07/30/13-12:41:20 (GMT) (tRAID): NOTE:  Qlogic coredump file written to '390VF4J:/tmp/QLogic_Coredump_port_0_390VF4J',rc 204E50, expected 204E50
07/30/13-12:41:20 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose returned -1
 
07/30/13-12:41:20 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
07/30/13-12:41:20 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed.  Stat f000
07/30/13-12:41:20 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State 4543
07/30/13-12:41:20 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
07/30/13-12:41:21 (GMT) (tRAID): WARN:  QLStartAdapter: ControllerErrorCount exceeds threshold.
07/30/13-12:41:21 (GMT) (tRAID): ERROR: QLInitializeDevice: QLStartAdapter failed
07/30/13-12:41:21 (GMT) (tRAID): ERROR: QLAddDevice: controller/device/chip initialization failed.
07/30/13-12:41:21 (GMT) (tRAID): ERROR: qlgEnableHostInterface: QLInitializeDevice failed.
07/30/13-12:41:21 (GMT) (tRAID): NOTE:  *************************************************
07/30/13-12:41:21 (GMT) (tRAID): NOTE:    QLogic Target Application, Version 2.0
1.08 6-13-2005 (W2K)
07/30/13-12:41:21 (GMT) (tRAID): NOTE:          iSCSI Target Application
07/30/13-12:41:21 (GMT) (tRAID): NOTE:   ******************************************
 
Exception: Data Abort
cpsr:  60000013   pc:  0x
All Replies
  • Hi,

    Have you found a solution to your problem?

    I have the same problem, and I can not find a solution.

    Thx

    Philippe

  • same issue for me.  If we bump it enough times someone is bound to look at it :)

  • Any luck with this?  Anyone?  I've got 5+ controllers with this issue! W T F!

  • Hello serv-adroit,

    If you are getting the same message as this then the controller is bad.

    07/30/13-12:40:50 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC

    07/30/13-12:40:50 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000

    07/30/13-12:40:50 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543

    07/30/13-12:40:50 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed

    07/30/13-12:40:52 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 0058bfa0 4c0c8 bytes , result 0

    What the issue is that the controller can’t startup as it is suppose to. Now you maybe can do a sysWipe on the controller and see if that will allow the controller to fully boot. A sysWipe in most cases requires a dell technician to do it as there is a risk of data loss. Since I am guessing that there is no warranty on your MD anymore then that is your call. But if you do a web search you can find the instructions listed on how to do it.

    Please let us know if you have any other questions.

    DELL-Sam L
    Dell | Social Outreach Services - Enterprise
    Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

  • It's my understanding that to run sysWipe it need to finish booting and command coube be typed at prompt.  Unfortunately it never gets to that point, just keep re-booting.