Lifecycle Controller - Firmware Update doesnt go past step 2

Servers

Servers
Information and ideas on Dell PowerEdge rack, tower and blade server solutions.

Lifecycle Controller - Firmware Update doesnt go past step 2

This question is not answered

Hi all

Having a odd issue since updating to the (currently) latest version of the Lifecycle Controller 2 (specifically LC2 v1.1.5.165).

Have been successfully using the Dell build of the SUU 7.2.1 ISO (and previous versions) hooked up via Virtual Media , but since I updated to this version of the Lifecycle Controller 2, on step 2 of Launch Firmware Update section where you pick the drive (in our case, the image you have attached), I can see it normally, and choose Next and it does the Verify Selection part with the progress window as normal, but then when it finishes, it stays at the same Step 2 section again.

I can then click Next and the same thing happens...

Everything is pretty current (given I've managed to put on this very latest Lifecycle Controller firmware (had to use the Dell Repository Manager and export the bundle I'd made up to a linux bootable iso from the selections - anything relating to the SUU options you could do resulted in a buggy thing where you couldnt continue through this step due to something wrong with the catalog file it made - either it couldnt verify the digital signature or something, or that it couldnt decompress the xml, which wasnt surprising as i dont think it put it in a cab file like it is on the SUU iso, but anyhow, i digress sorry........) and I can view the firmware versions in the LC using View Current Versions and it will show me. Here's some fyi:

BIOS @ v1.6.0 (did this safely using a good old DOS boot disc via virtual media etc...never fails :D)
Diagnositics @ 0 (one of the reasons why i'm trying to just use whatever version is on the SUU for this)
IDRAC @ 1.40.40 (latest firmware for the iDRAC7 i could find - updated via the DRAC Web as per normal)
Lifecycle Controller @ 1.1.5.165 (latest i could find and newer than on the SUU 7.2.1 iso - updated as mentioned using the method I mentioned in the paragraph blub above)
System CPLD @ 1.0.3
and OS Drivers Pack @ 0 (dont think it matters to this process in question - we dont use the OS deployment thingy in the Lifecycle Controller as these are VMware 5.1 hosts so no need to etc)

Oh, and the server in question is a PE R720 :)

Not sure if its just that the Lifecycle Controller is newer than the ISO and thus ignoring its catalog, or that the format has changed slightly in the new one, and thus why its not recognising it as such.

Note that there are no errors in this, it just stays on that Step 2 window after its scanned the drive (ISO) like it hadnt even done it.

Things I've tried are pretty much turn the server off completely, and use the Hardware Configuration > Delete Configuration and Reset Defaults (as that clears Lifecycle Controller Configs as well as alot of other stuff - in the thinking it was something corrupt somewhere in there that needed a clearing).

Any ideas or thoughts would be welcome.

Certainly not a show stopper but I'd like to know if its going to be the case for the other servers once I get this one configured and into production - I guess I'm being precautionary too, as it would be a bit of a pain if for some reason this avenue is now faulty and would stop me using this method to update the firmware of the other components easily for sure.

Cheers

 

All Replies
  • Tarquel,

    Download these 2 ISO's and burn them to disk -

    Server Update Utility (7.2.1)(SUU) - www.dell.com/.../poweredge-r720

    and

    Build and Update Utility - www.dell.com/.../poweredge-r720

    After those are burned then you can boot to the BUU disk and then once at the menu swap out the disk with the SUU. From there select the Firmware (Platform) update. That will run the update outside of the OS and will do the entire server.

    Let me know if this stabilizes the system and gets it current.

    Chris Hawk

    Dell | Social Outreach Services - Enterprise
    Get Support on Twitter @DellCaresPro 
    Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

  • Hello Tarquel,

     

    Your firmware seems to be updated to latest version. Can you unmount and mount the iso image? Also is the problem seen consistently on the same platform?

     

    Regards,

    Balaji

  • I am experiencing the same problem (not through virtual console), since the end of May, trouble-shooting with Dell techs and there is no resolution yet that I know of. We have had motherboards replaced, and it would work for one try, but come back again next try. We got a replacement server after that and it worked fine.

    A couple weeks later, I ran into it again on 2 servers. I trouble-shot and ran LCC repair packages, and much more, with no change. We had gotten replacements that came up with the same issue. Even the replacements for the replacements came up with the same issue. Still trouble-shooting with Dell techs.

    I typically update via the FTP site, but I have noticed that it doesn't work even when I point to an SUU DVD (various versions 7.20, 7.21, brand new 7.30). Looks like it is working, but the Select Updates window (Step 3) never appears.

    Also, when you click Exit and reboot (in the top right corner of the LCC), the cursor turns red (LCC crash) and the system hangs until powered off manually. Very frustrating.

     

  • What is the actual name of the iso file that you downloaded from the support site? Is it SUU_730_x64_Q22013_A00.iso?

     

  • No it the 32 bit version of that file. I have been told by Dell technicians that the 64 bit files do not work with the LCC.

    SUU_730_x32_Q22013_A00.iso

  • Excellent, that's one of the trick questions. They are correct, the 64 bit will not work. What exactly is attached to this system, KVM sips, any cards in the slots, anything on the USB or serial ports?  Can you walk us through the steps that you are taking to reproduce the issue so that we can do so here as well?

  • Directly hooked up to a monitor, keyboard, and mouse (per technician request), or through a non-IP KVM, it happens. There are Broadcom 57810 cards in the PCI slots, but it also happened after I removed them. Nothing on serial. PS2 to USB adapter for the KVM in the USB ports.

    F10 to enter LCC at boot. Select update firmware, point it to FTP site, or DVD, step 2 reaches out and verifies catalog signature (as per normal), pop-ups disappear and when Step 3 (Select Updates) window should appear, nothing happens.

    In the meantime, my company has gotten plenty of other identically configured servers in for other orders and we use the same procedure for them and they work fine (same cables, KVM's, network, etc..).

  • Have any of the replacement systems been captured on this end, and if so, can you give me the first name of the person that captured them?

  • Yes. Jason captured them. I believe he has 2 in Texas and is looking to get one of our other replacaments to him as well. He claims that everything works fine when he tries them. He keeps syaing that everyone says it is environmental, but we have some that fine in the same environment with the same configuration and firmware levels. We have had a Dell tecnical representative here on multiple occasions and he is coming back tomorrow to continue. These systems have been rolled back and brought up to the latest versions many times, but the LCC firmware update process is still not behaving consistently. Once in a while after doing some different things (lcc wipe, board replacement, card removal, etc..) it will actually work, but if I try it 5 minutes later, it is back.

  • Sounds like we may have a timing issue then. On the systems that work, do they always work 100% of the time, or are they deployed and aren't retested? Are your idrac ports connected to a network or do you do shared network ports?

  • The ones that work are consistent. We do ship them to customers after testing and configuration.

    The iDRAC is connected to the network only when we need to use it. We basically go in to clear logs and thats about it.

  • Hi jcartert,

    Can you provide us the service tag and configuration details of the failed systems?

    Regards,

    Balaji

  • Seeing the same issue. Not going after step 2. Same as jcartert we also see the red cursor after that when exiting the LC.

    Jcartert: did you manage to solve this issue yet with Dell?

  • Hi guys.

    Meant to respond back on this issue but looked at this in great detail over many weeks with some Dell Enterprise Technical Support folks.

    It seems to be some conflict / issue with LCC, the 10Gb Broadcom chipset cards, and the way the CSIOR  [inventory collection using the LCC process] works - it doesnt in this case and bombs out.... you can see this in some instances where leaving the LCC without a reboot at the end of it will then cause some sort of red error exception. It only happens with the 10Gb card in, using the later LCC firmware, and with CSIOR enabled as I recall.

    I think its from version 1.1.5.165 onwards, although I believe the next LCC firmware update due December time will have the issue fixed.

    What've done is to stick at LCC 1.1.1.8, but update everything else and i've stopped using the SUU for now, and just doing it via the iDRAC7 interface - for those that dont know, in this version of iDRAC, you can update the firmware of things using this method using the normal win32 DUP installers - just go to the Update and Rollback section and upload and apply the DUP's as needed :)

    Sorry its not highly detailed, but i got heavily delayed due to it and other reasons, so I cant really go into it in work time presently.

    I pointed Dell tech's to my thread too, so its possible they are watching this thread too :)

    EDIT: a bit more info that just came back to me is that the reason it doesnt go past step 2 is that, to me, it is trying to inventory in order to the comparison of current and available firmware, and cant complete as the inventory process bugs out.

    EDIT: additionally, when the guys were trying to replicate it, it wasnt happening on their setup, but I did notice that my R720 in question had a different revision of motherboard to the one they were using, so that helped narrow things down too.

    Cheers.

  • We don't have 10Gb Broadcoms installed, only 1Gb broadcom and we still see the same issue. There are fibre channel cards in the servers though.

    Two out of three servers did update fine, but the third didn't. Same hardware, same firmwares.

    Dell is replacing the mainboard by the end of the day. Hopefully it will fix this 'issue' for now.

    Keep you posted.