After driver update, OME still shows update as available - Dell OpenManage Essentials - Systems Management - Dell Community
Systems Management Forums

After driver update, OME still shows update as available

Systems Management

Systems Management
Dell Systems Management Solutions: Dell OpenManage, iDRAC, Repository Manager, Microsoft SCCM, Chassis Managment Controller, and more

After driver update, OME still shows update as available

  • Hi,

    Just upgraded to OME 2.0 and downloaded the latest catalog.  I'm trying to apply a PERC H710 Mini driver to an R620.  OME shows the current driver version as 6.802.19.0 with 6.802.21.00 as available.  I queue the update, the server automatically reboots, and OME System Update Summary page shows the task as completing successfully.  I re-run discovery/inventory/status and OME still shows that same update as available under the Non-Compliant tab.

    I checked OMSA (v7.4.0 and Inventory Collector Agent v7.4.1) and iDrac7 web interfaces and they both report the driver has been upgraded to 6.802.21.00.  Seems like part of OME isn't getting the message the driver has been upgraded.  Attempting this process again returns the same results as above.

    Please advise.  Thanks.

    Ryan

  • Hmmm, ok it usually takes 15 or 20 minutes for the inventory to update.  But it seems you may have waited this long.

    What about when you look at the inventory details from the device tree and clicking on the server...are the versions the same? (old?)

    Thanks,

    Rob

    delltechcenter.com/ome

    DELL-Rob C
    Social Media Support
    #IWork4Dell

  • Rob,


    Does it have to be 15-20 minutes?  These days people want instant gratification.  Someone trying out OME for the first time is going to be confused when OME says the updates just installed need to be installed again.  Admins that don't trust updates enough to run them over night will have a frustrating experience trying to apply updates in a more interactive manner.

  • A forced inventory for me on a single device usually takes 30-45 seconds.

    To the original poster - Did you run inventory on just that device or your whole range?

    If just that device and the inventory completed and you still see different data than the DRAC I would delete that device and rediscover the range it's in.

  • Please consider this analogy.  Say Windows Update pops up on your PC and asks to install 5 updates.  You allow it to install the updates and then reboot when it asks.  After the restart, you open Windows Update and it shows the same 5 updates as needing to be installed.  This is just not a good user experience.

    I've now accepted that after I deploy a batch of updates, I'll just have to wait until the next day to see if OME will stop offering them.  This has caused the task of updating our servers to grow from a weekend project to a several month project (over many weekends) because I want to test the updates on less critical servers first.

    Waiting 20 minutes doesn't work for me. (I waited all day Saturday once.)  I've tried forced discovery/inventory with mixed results.

    This whole thing contrasts with Dell's Server Update Utility.  I run SUU and it installs updates.  I run it again and it shows no updates needed.

    Physiologically speaking, seeing a system move from the non-compliant tab to the compliant tab will cause the reward center of the brain to release a little bit of dopamine.  This game that OME plays has the opposite effect of causing frustration when the expected reward does not materialize.

  • Well put, MK1024.

    Rob, apologies for the delayed response.  For some reason I wasn't receiving email alerts that someone had responded to this post until this morning.

    I was aware it could take several minutes for this information to update in OME so I was sure to wait long enough.  Further tests yielded mixed results.  Sometimes OME would finally register the updated information after a few hours, in another case, I waited until the following morning before it showed those updates were no longer needed.  In no case did OME update promptly after manually running an inventory.

    MK1024, I agree that waiting 15 - 20 minutes is too long, but, as I understand it, the inventory collector agent which is installed on each server with OMSA does not run on-demand.  This is what inventories the server to collect components, driver versions, etc and populates an XML file which OME then retrieves and uses to update itself.  The inventory collector only runs on reboot (or if you manually restart all OMSA services), but, this isn't explained in any documentation.  If OME continues to check-in before the inventory collector agent updates the XML file, OME will remain unaware of any updates.  It certainly seems to be a bit of a clunky mechanism and can easily make for a frustrating experience.


    Cameron, I ran the inventory on just particular devices, but, entire ranges are re-inventoried nightly on schedule.  I've seen the recommendation of deleting and re-adding devices talked about for other various issues on this forum, but, for a production environment I don't feel this is a sustainable solution.  Having to delete/re-add devices each time we push out updates can't now become another task to manage.  Rather we just need the software to work as advertised.

    It's worth noting that I did not experience this problem in any past versions of OME.  If I rebooted (or manually restarted OMSA services) and then kicked off a manual re-inventory of a specific device, OME would accurately report all updates and version as soon as the re-inventory completed.

    Rob, I realize it might be tough to troubleshoot further.  Could you maybe just relay this experience as an FYI to the OME engineers?  If it continues to hinder my workflow, I'll probably need to call in.  And thanks for your efforts.  You do a heck of a job managing these forums.

    Ryan

  • Thanks for the well thought out remarks.  I'll be sure to pass them along.

    I can't recall if you do mostly in band (OMSA) updates or out of band (WSMan).  

    The 15 or 20 minute waiting period is to mitigate the fact that we need 'some time' to ensure the updates complete on managed node.  Folks might kick off inventory manually before the 20 minutes with some success.  For in-band, there may be an OMSA issue where the managed node needs to restart the OMSA service to get the inventory updated.  This might happen for updates where a reboot is not required.  Perhaps SUU does not have this issues since it can control the OMSA service re-start?  Just speculation on my part.

    So a manual reboot of the MN or a restart of the OMSA service might be a workaround.  I'll try to find more info on this particular OMSA issue and confirm if I'm correct.

    But I would be interested to know if a reboot or restart of the OMSA service does anything to improve the situation for you (not that that is a final solution...just for data gathering purposes).

    Best,

    Rob

    DELL-Rob C
    Social Media Support
    #IWork4Dell

  • Ryan,

    Thanks, I think you might have stated the OMSA/inventory explanation better than I did.  I'll send the thread to the team to try and dig a bit more.  Perhaps it is some combination of the OMSA timing thing and something else on our side.  Let's see what I can come up with.

    If you've not mentioned already higher up in the thread, you might let me know what kind of DUPS you see this more frequently on (if there is a pattern).

    Thx!

    Rob

    DELL-Rob C
    Social Media Support
    #IWork4Dell

  • I do all in band (OMSA) updates.  Now that I better know the role that OMSA plays (thanks Ryan), I will test some different scenarios.

    Maybe what's needed is for some functionality to be added to OMSA to better support OME.  If OME were able to cause an on-demand collection, that would give the OME engineers more to work with.

  • .  

  • Hi Rob.

    I am having a same problem that after update my BIOS and Firmware updates on the PowerEdge servers through OME, after couple days OME is asking to updates the same thing again? There seem to be a bug in the tool. Do you have any ETA and confirmation that the problem is in the tool itself rather than our environment?

    Thanks

  • I did some testing this morning.

    1. Used OME to install Network_Firmware_NF92Y_WN32_7.10.18.EXE on a PowerEdge R300.  OME completes the install and reports success. The update required a reboot and OME successfully rebooted the system.

    2. I wait 20 minutes and the update is still showing in OME as being needed. I wait an hour and OME is still showing the update as being needed.

    3. I manually reboot the updated server. 

    4. After the server comes back up, I run Perform Discover and Inventory Now from Manage > Discovery and Inventory.

    5. I check the System Update tab after OME says the discovery and inventory task is 100% complete and the update is still showing as needed.

    6. I run Refresh Inventory from the Devices tab.

    7. I check the System Update tab after OME says the inventory task is 100% complete and the update is still showing as needed.

    8. I restart DSM BMU SOL Proxy.  No change.

    9. I restart DSM Essentials DA Service.  No change.

    10. I restart DSM Essentials Host Service.  No change.

    11. I restart DSM Essentials Network Monitor.  No change.

    12. I restart DSM Essentials Task Manager.  No change.

    13. I close my web browser and restart using the Essentials icon from the desktop.  The system is now listed as a compliant system.


    The last time I ran into this problem, I restarted all the DSM Essential services and the web browser and that allowed OME to start seeing the correct status.  This time I was trying to see if restarting any individual service resolved the problem.  Next time, I'll get the web browser restart in the process earlier.  

  • After you update the inventory,  you need to double check whether the update has gone through.  

    In OME,  Check Details of the device to see whether it show new or old firmware/driver.  You can then check Lifecycle controller log to see whether the update has actually happened,  success or failed?  

    It happened to me several times.  In every instance,  OME reported the inventory correctly.  There is some firmware issue. 

    Check this thread:

    http://en.community.dell.com/techcenter/systems-management/f/4494/t/19599861

     

  • Ideally, if an update fails to install, OME should report that the update failed.  It shouldn't be necessary to manually check updates if OME is showing success.  In my latest test, if I manually run the DUP on the system, it does show that the package version is the same as the installed version.

  • There is some problem with some firmware/driver, for example Power supply firmware.  The new package contains old version.  When you update the server,  the system just updated with old version successfully.  OME will not treat this as error.   When the inventory is updated,  OME will correctly report the old version, as current version, and update is needed.

    For example:

    Successfully updated [PSU-1] PWR SPLY,750WP,RDNT,FLX, 69.45.9B, A00.

    Updating Flex AC750W Platinum PSU(PSU PN: N30P9), 6B.47.9B, A00