I have 2 2950's and an R420 that are all exhibiting a similar issue. I had OMSA setup and working just fine before, with 1 2950 running ESXi 4.1, and the other 2 running ESXi 5.0, and the appropariate Dell OM-SrvAdmin vibs installed. I had OMSA 7.1 installed on a windows 2003 box, and I was able to access them just fine.
I have recently upgraded all servers to ESXi 5.1, and even after following the instructions to reinstall the OM-SrvAdmin after the upgrade, I am unable to connect to any upgraded ESXi server. I first tried to install OM-SrvAdmin-Dell-Web-7.2.0-6870.VIB-ESX51i.zip on the ESXi server, but found I could not connect. I thought perhaps it was an OMSA version difference (the Win2003 box had 7.1 installed), so I upgraded it to OMSA 7.2. Still couldn't connect. I then tried to update the ESXi host to a different version, I've tried both OM-SrvAdmin-Dell-Web-7.1.0-6339.VIB-ESX51i_A00.zip and OM-SrvAdmin-Dell-Web-7.1.0-5304.VIB-ESX50i_A00.zip. I've made sure to be in maintenance mode during the vib install, and a reboot after the install. Nothing that I've tried seems to get OMSA connectivity back ... every time I attempt to login, after clicking on "Submit" it quickly jumps to the error screen showing "connection error". I even followed the directions from a thread I found (which I remembered from before) where you add the "cipherList" node in the appropriate spot in the /etc/vmware/hostd/config.xml file, to allow connections from OMSA on Server 2003 ... still didn't work.
I'm pretty sure that there's nothing wrong with the installation on the Win2003 machine, because before I upgraded the last 2950, I could still connect to it when it was running ESXi 5.0, even after I upgraded to OMSA 7.2 on the windows box.
Does anyone have any hints? Anyone else having trouble connecting to OM on ESXi after upgrading to 5.1? Any hints on how I can try to debug this ... I can't really find much for logs or information on what might be going wrong here. Very frustrating ...
OK, perhaps no one else is having this issue. Well, it turns out ... this may only be an issue with Windows Server 2003. I installed OMSA on a Win7 machine, and it can connect to all of my upgraded ESXi 5.1. servers. In fact, I had OMSA 7.1 installed, and it connected to my server that had the OM7.2 VIB installed, but then told me that I had to upgrade OMSA to 7.2 to work with it (far better than an unspecific "connection error").
The knowledgebase article I mentioned was from VMWare: VMware KB: ESXi 5.0 disables nonsecure ciphers in Internet Explorer 6 . This worked for Windows Server 2003 connecting to ESXi 5.0 (has little to do with IE6, because I had IE8 but it still wouldn't work without doing these steps) ... but this same fix does not appear to be working for ESXi 5.1. Anyone have any thoughts? Really would like to still connect from a Server 2003 box.