I updated the bios on a C1100 CS24-TY cloud server and now it will not POST - turns on, fans spin up full blast, but no POST and BMC is not available. The service light on the front is also orange.
I found a similar post: http://en.community.dell.com/support-forums/servers/f/956/p/19491173/20298901.aspx
talking about using the bios recovery mode (Jumper) using AMIBOOT.ROM on a USB flash drive, but that does not seem to be working for me. Does anyone have any experience doing this? anything else i can try?
*** BEWARE ***
*** YOUR MILEAGE MAY VARY AND YOU'LL MORE THAN LIKELY BRICK YOUR SERVER ***
*** CONTACT DELL SUPPORT ***
Somehow I cannot find my post referencing to those 3 jumpers =D.
In short its like the 3 sea shells... Everyone seems to know what they do but us newbies.
When I bricked mine, then there was a mention about a recovery jumper, which I couldn't find in any public documentation, so I started with the cluster of 3 jumpers near J22, moved 1st to 1-2, power on, moved 2th to 1-2, power on, etc, until the fan start to go full blast.
But you seem to be in luck if you fan are already doing it...
It does take a long while for the boot strap BIOS to show up (5 minutes and more). so connect a monitor and a USB keyboard and wait.
PS: Mine is showing up as Product: CS24-TY, BIOS S99C3B22 works in it and 1.79 for the iBMC.
I haven't tried B23 or 1.8x yet...
Also there maybe a massive issue with machines with a 3Axx BIOS, I'm trying to figure out if the first one I bricked and recovered wasn't one of those. I have 11 more to do :( At least 1.79 seems to works with 3A16 and 3A19.
Now why can't I find any archived 3Axx BIOS...
Is you server says CS24-TY on the Dell badge (top, right side of the server)?
If so, while it look, taste, smell, sounds like a C1100 =D It is a special build.
Most of us having issue with "C1100" ended up with those.
And, I think, there is no support for second hand special built.
... Now how the hell I got 3B22 to work on one of my CS24-TY =D
I wanted to chime in and cross link a few threads with some info I thought might help people. This thread seems "related" in a "set of four". I am going to post this same post on all four in an effort to help folks...
There are a ton of the C1100/CS-24's out there on eBay right now, and I am yet another on of the people who just couldn't say no and got one to user as a Hyper-V 2012 box (the no UI version). It just seems like about the best deal going to add some raw horsepower to your existing virtual environment (which I have at home because I am a geek and just enjoy this stuff...)
What thee threads seemed to my saying to me (IMHO) is that there seems to be some differences in the machines out there, and that all C1100/CS24's were not created equal. What I decided was going to be my personal indicator of "the good ones versus the risky ones" was would they take the most current (at the time of this post) C1100 bios from the Dell site (B23) and BMC firmware (1.82). I contacted a couple of the sellers on eBay to see if anyone knew about this, and (NO, I DON'T WORK FOR THESE GUYS!) I ended up talking with Peter Neiman from one of his eBay listings. Hopefully the moderator doesn't chip that link, because the guy earned the right to be specifically mentioned in my mind. He pulled one of the servers I was looking at and updated it to the most recent version to make sure it would work. Needless to say, not having to risk spending another couple hundred bucks to replace a board that fails to update as some people are seeing is worth a ton, so I bought mine from him. Again, I don't work for these guys, and I'm not saying you have to buy from them, but whoever you buy from, it might be worth asking about BIOS and F/W before you buy. That's my $0.02.
Hope I helped someone,
also any ideas why one system has fans running at full throttle at 10000 RPM even when idle in the bios? what can i do to dethrottle it?
this system has 1.47 bmc and 3A16 bios.
two other system are fine!
Also it seems one system had 1.29 bmc and 09 bios, which i updated to 10 version. its working fine, i also see qpi errors on bmc log. it seems bmc 1.47 started flashing yellow maintenance led due to qpi errors. i disabled qpi error reporting from bios to avoid this.
im running ubuntu 13.04 and kernel 3.8.5. has anyone seen these qpi errors in bmc system events log?
Take the first 6 characters from the bottom of your ASPEED BMC chip here:
Support / Downloads (Select Product Model: QSSC-S99K 1U / 2U): http://www.quantaqct.com/en/02_support/02_download.php?page=10&actions=1&xauthorise=77
DELL's Support / Downloads: (See Previous Versions ... FYI ONLY ... You may Brick your machine if you have a Quanta Custom Solution.)
MAKE SURE You get the right Firmware for your board & chips!!
There are several versions of this animal so make sure you do a complete hardware investigation.In Linux terminal: lshw ... as su / root will give an exhaustive list. To dump it in a text file ( .txt ext. optional... Add it for Winblows compatibility ): lshw > hwlist.txt
For those of you scratching your head trying to install a desktop OS on these... They are Enterprise action *** servers so don't waste your time. Most current Desktop distros will NOT work unless you Virtualize and VNC them on a Enterprise action *** OS. KVM or Xen on Redhat, CentOS, Suse EL / openSUSE, Etc... All of These will work with any BIOS / hardware you have (Base Installs w/o Desktop environment; DE's mileage varies by BMC ver. ) Of course VMware & Xenserver will work too.However; Macpup 528 works with it's lightweight Enlightenment desktop, as do older DE's like gnome2 on ubuntu lucid 10.4 LTS.The key to getting a Desktop to run ootb is to use Lightweights or versions prior to the world upsetting new gnome 3 and KDE4 releasesRoll your own from X, or... Just use the CentOS-6.4-x86_64-LiveDVD.iso which will boot into a desktop akin to gnome2 w/o any problems.
Sorry guys, Windows just won't work on some of these w/o the right drivers and/or BMC firmware. Ent Server 03 & 08 crash - BSOD during install with some of these machines with the buggy firmware.
The problem with these is the ASPEED - BMC Chip (2D Video only) Not the BIOS:http://www.aspeedtech.com/products.php?fPath=20&rId=194http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface
========================================================QUANTA Computer Inc. BIOS_3A03 RELEASE NOTES for Z99 (DELL S99)========================================================Project Name : Z99BIOS Version : Z99_3A03Build Date : 2011/08/17Update BootBlock: YESClear CMOS : NOClear NVRAM : NOChecksum : 32A85522H
=========================================== HARDWARE REQUIREMENTS/REVISIONS===========================================System hardware revision supported: Z99 MBTylersburg C2 version for Westmere-EP support is required.
=========================================== SYSTEM FIRMWARE REQUIREMENTS/REVISIONS===========================================BMC : 0.85FRU and SDR Package : N/AVGA (embedded in AST2050) : 0.90.7c (2009/10/29)Intel(R) ICH10R SATA RAID : Production Version 126.96.36.1992Intel(R) 82576EB (ZOAR) : 1.3.28Intel(R) iSCSI BIOS : 2.2.58Intel(R) SPS F/W (ME) : SPS_01.01.02.007Intel(R) Thurley Reference Code : p2.97 (June,24, 2010) - CPU RC Version : 1.08 - QPI RC Version : 1.85 - Memory RC Version : 2.30
========================================== IMPORTANT INSTALLATION NOTES==========================================WARNING: It is very important to follow the flash option provided in the batchfile (F.BAT) to update BIOS. Uisng incorrect flash option to flash BIOS maycause damage to your system.
BIOS update utility (AFUDOS v4.31):(1) Copy AFUDOS.EXE, BIOS ROM and batch file to a HD or USB Flash Drive, execute F.BAT under DOS environment and update finishes automatically.(2) Reboot system and new BIOS runs.
Note: AFUDOS will not update ME F/W.
======================================== BIOS RECOVERY INSTRUCTIONS ========================================- Recovery process can be initiated by setting the recovery jumper.- Recovery process will not flash boot block and NVRAM.- BIOS recovery can be accomplished from SATA CD and USB Mass Storage device. - The recovery media must contain the image file 'AMIBOOT.ROM' under the root directory. - The following steps illustrate the recovery process: (1) Turn off system and insert recovery media. (2) Set the recovery jumper from 1-2 to 2-3. (3) Restore the recovery jumper to normal position 1-2. (4) Turn on system and BIOS will start recovery process. (5) The BIOS POST screen will appear and display the progress if the boot block video is supported. (6) Once the recovery is completed, 4 beeps would be heard to indicate recovery process successful and system will reboot with new BIOS.
Note:(a) The jumper should be restored to normal position 1-2 before turn on system. Otherwise, recovery process may be invoked again after reboot.
======================================== KNOWN ISSUES/WORKAROUNDS========================================- Only B3 stepping of SAS controller off riser card could be detected by PCIE GEN2 bridge.
THIS IS NOT THE COMPLETE FILE.
The rest of the story and more drivers can be found here (inside the QSSC-S99K 1U BIOS 3A03 zip):Quanta Support / Downloads (Select Product Model: QSSC-S99K 1U / 2U): http://www.quantaqct.com/en/02_support/02_download.php?page=10&actions=1&xauthorise=77
Managing the Server Via a Remote Console (Page 7-8)PowerEdge C servers offer, as standard, many options of remotely managing theserver. All of the following methods will effectively allow configuration of BIOS setupoptions and full OS installation.
• iKVM: Virtual KVM over the network, provided by the BMC. Access via “remote control” inside the BMC Web GUI. Requires Oracle Java on the client machine.• SOL: Serial Over LAN. Fully virtualized serial interface—handled by the BMC. Requires IPMItool.• Serial port: As above, but over a real physical serial port, so it can be redirected via a concentrator.
Notes Regarding Serial ConsoleUsing the serial port can be an efficient means of managing your server, but it doesrequire some work to set up.Parameters must be configured into the Linux bootloader and provided as a kernelcommand line parameter for a smooth handoff. (Otherwise, output will just cease).You must ensure that the serial port setup (data rate, stop bits, etc.) matchesbetween:• The console itself: a terminal emulator such as PuTTY, or an instance of “ipmitool sol activate”• The server’s BIOS setup• The Linux bootloader (likely “grub”)• The Linux kernel itself (via command line parameter “console=”)• The “getty” instances referenced in /etc/inittab
Expect to have something like this appended to the Linux kernel command line (viathe bootloader):console=tty0 console=ttyS1,19200n1Note that multiple “console=” entries are allowed—the last one listed sees the startupmessages (stdin/stdout/stderr of /sbin/init).
7Dell PowerEdge C System Management Quick Start GuideUsing iKVMiKVM offers a virtual console for your PowerEdge C server over the network. Here are the steps to try it out:• Install Java on your client machine.• Point a Web browser at the BMC’s IP address (https://my.bmc.ip.addr).• Log in (defaults are root / root).• Click “console” tab.• Click “launch kvm” or “launch virtual console.”You can manage the BIOS options, install an OS interactively, and so on.
Virtual media can also be mounted—media local to your client machine will appear as media local to the server, across the network.It is easiest to enable “full keyboard,” so that all keys are captured and passed to theserver:PowerEdge C1100/C2100:keystrokes to targetAll other models:Tools | Session Options | Pass allKeyboard | Full KeyboardPasting into the iKVM window is available for C5125 / C5220, but copying from thewindow is not possible. (This is a fundamental limitation: Remotely, keystrokes andmouse movements are being emulated by the BMC at the hardware level. Thistradeoff allows the use of any, or no, OS).
PDF GUIDE: Dell PowerEdge C System Management << Do a Google Search with it for more.http://188.8.131.52/pec/files/QSG_PowerEdge_C_System_Management.pdf
Hi Robert, really helpful and thorough information you posted!
I seem to have a Quanta machine, based upon looking up "00269E" from my ASPEED BMC chip.
Successfully flashed the (QSSC-S99K 1U) with BIOS version Z99_3A03 and BMC with v1.02 from the Quanta site.
However I cannot enter into the BIOS setup menus for the life of me. System ignores the keyboard (F2/F8/ctrl-S, etc) at boot. I cleared the CMOS, tried via KVM, different keyboards, etc. I could get into setup prior to these versions.
Any recommendations? I notice the 3A03 BIOS release notes says BMC 0.85, which differs from 1.02 on the site... could that be it?
Thanks a lot!
(TIP: Use pause/break key to freeze screen / messages / POST ... to read any errors. same key or any to continue -- [if keys will work ])
On my server, the USB ports were rusty and required using 1000 grit (Black) sandpaper rolled to fit and clean the ports before anything would work in them. Perhaps it is a bad port ... clean and try again, also try connecting keyboard to other ports.It has been my experience that sometimes not all ports will work during POST.
Still no dice? Is the orange light on the front blinking? (Yes = Problem detected and recorded in System Event Log) Can you access the Remote Management Controller (BMC) through the maintenance port? (Should default to DHCP - scan network / look at router / DHCP server client list for that MAC you listed. )If so what does it look like (Blue/DELL or some other name?) and are there any errors recorded in the system event log?
BIOS and BMC Communication information begin on page 79 of the Quanta User's Manual. Is the green LED near the BMC blinking? (Yes = Heartbeat, BMC System up and OK.)
IPMI commands are also supported from another machine to attempt communicating with the BMC.
Caveat is that BMC access is controlled by the BIOS (Usually AMI)
To find an update, if there is one out for it you would need to contact DFS Support at 800-891-8595... and possibly Quanta.
Further verification beyond the aspeed chip is advised.
Sorry if I didn't convey that clearly.
There is no need to upgrade unless there is a problem.
Flashing instructions in the Release Notes should be followed when you do flash and make sure you are flashing the right chip on the right MB...
There is a BIOS and BMC update as you are aware... The BMC Release Notes state that the Quanta updates are for:"Hardware Platform: S99 Rev. E (or later)"
The DELL (intel) MB number is under the CPU1 heatsink (if I recall correctly) ... remove memory to peek under heatsink with a flashlight ... Do not remove heatsink or cpu. This will give you the original MB number... Stickers on the board and chassis will give you the custom builder numbers / modifications.
If you cannot get any keyboard to work, then I would assume you have the wrong firmware / locale ... wrong keyboard map.
Perhaps Quanta changed something when their support for these machines disappeared for a spell recently.In which case I would contact them directly to inquire whether the files although in English on the website are coded for Chinese keyboards. - Just a guess at this point as anything is possible.
Dell may be able to shed some light as well. - Have your Service Tag, BIOS, BMC, & MB sticker info handy.
Ultimately, let this be a lesson to others that you MUST do your due diligence in researching your hardware and Verify with DELL and/or Quanta that you do in fact have the right firmware Before flashing. This has been the mantra of many postings all over the web and here regarding these machines. Look before you leap.
At the system boot, BIOS should perform initial diagnostic test of hardware (CPU, Memory, PCI ...etc). If "BIOS Detected Error", BIOS stops the system. In any case, display an error message with POST Code. Some of the event will be logged to BMC (e.g. CMOS checksum error, Memory ECC error, PCI error...), in any POST stop case, BMC will timeout with a POST code event log.
BIOS stratagem is to stop the POST and display error message only when the error is detect by BIOS during POST. The error detected by BMC or MMB during POST will handle and react by BMC or MMB, these errors are unaware for BIOS, so BIOS will not stop the POST and not display error message to screen. For example CPU temp is monitored by BMC to log the event, but no reaction by BMC, the CPU will do TM2 (Thermal Monitor 2) to reduce the CPU’s speed and the temp, the worst case is, if CPU’s temp go more higher, the CPU’s Thermtrip output pin will assert to ICH10 and make a system shut down by H/W (not by BMC). CPU’s voltage is also monitored by BMC with event log,
If error is detected, POST Code is displayed on Screen.
The BIOS provides the hot keys during POST or BIOS Setup as following:
Note: When populate add on card, please don’t press add on card option ROM control key and “F2” (Enter BIOS setup) at the same POST. Because some add on card may need to be modified first and the information will be show in bios setup menu at next boot (for example: SAS Raid card HDD information).
(removed duplicate post)
Thank you so much Robert. Here is the info I found per your tips:
Although it shows up a Quanta based on the http://www.macvendorlookup.com/ link you provided, was/is the Quanta BIOS Z99_3A03 the correct bios?
I cleared the CMOS using the jumper and now it won't boot... that made it worse. Getting "Error(0005): CMOS Checksum Bad. Error (0150): Processors failed BIST. Fatal Error... System Halted." Repeated restarts or CMOS clears don't get beyond the halt. I'm hoping BIST error is specious and a symptom of a wrong BIOS/CMOS. I've barely run the system (more than a few minutes at a time), so even with wrong settings unlikely to overheat or damage.
The AMIBOOT.ROM flash procedure with the jumpers (that worked previously to get it onto the Quanta bios) no longer works. Stays stuck on "Reading file AMIBOOT.ROM" with old or new bios files. I am unable to access via BMC... no show on network. Tried rebooting, letting it sit, etc, many times. System is really stubborn now.
Maybe I need another BIOS file?
Processors failed BIST:http://www.intel.com/support/motherboards/server/sb/CS-000208.htm
CMOS Checksum Bad:http://www.tech-faq.com/cmos-checksum-bad-error.html
Remove AC power to the system for10 seconds and restart the system.If the problem persists, see "GettingHelp" on page 123...
Pg. 123: For customers in the United States, call 800-WWW-DELL (800-999-3355).Go to dell.com/contactdell.
Remove AC power to the system for 10seconds and restart the system.
Contact Quanta with the info you posted above. That is a Quanta machine. Their firmware is Supposed to work in their machines.
I don't know what you've done or how you got there, but there are several options to flash, one of which is through the BMC admin page and it is extremely easy to brick a machine if you do not know what you are doing.
My firmware version is: 1.55.50361 so I did not flash mine as it would be downgrading.I only posted all of this information to help others on their journey. It's up to you to avoid pitfalls.
Many problems that send people hunting for firmware are actually due to their choice of OS install and subsequent settings within the BIOS and OS.These are Custom Purpose built machines that were taken out of service due to contract SLA's. They will only work properly with certain supported OS' which are clearly listed in both Dell and Quanta's user manuals.
I found that bumping down the QPI speed a notch and using CentOS (redhat) eliminated the orange light and all other problems. I use a minimal base Qemu/KVM virtualization install and virtualize everything else.I've even ran FreeNAS successfully on these... but later decided it was a waste of 16 threads of horsepower.With twin Xeon e5630's and 72GB ECC in my machines I went solely with Virtualization and left NAS to lesser machines. Winblows server and XP works fine virtualized. Seg your services into VM's.
Unfortunately if you are not extremely careful navigating information and instructions, you will step on a land mine... which you apparently may have done with a bad flash.
You may be able to get Quanta's help via email:http://www.quantaqct.com/en/04_contact/01_inquiry.php?ffid=&type=0
And try DELL's support as well.
ONLY the vendors know what is truly compatible with your hardware.That is why I recommend due diligence in hardware snooping and recording the information you gather so that you are fully armed for any questions that the support agent will throw at you.Another tip is to get a case number and call back to ask that your case to be escalated to tier 2 or above... as tier 1 support calls usually go to some idiot in India reading How-To manuals from a screen and have you dancing for an hour with step-by-step stupid tricks covering ground we've already covered.
You will usually not get anyone with any experience on your first or second phone calls and certainly not until your case is escalated to at least tier 2 where you might find someone with half a brain and better English.
The Real DELL techs reside in tiers 3 and 4. ... Tier 1 = India call center. Tiers 2 and above = USA.I worked as a HP/Compaq TSE (Tech. Support Engineer) years ago (1999-2000 - Y2K Issues), so I know first hand how USA support centers work.Back then everyone was shifting tier1 to India, now it is almost certain your first call will land there. As far as Quanta in China... Who knows what you will find... probably the same thing.
Perhaps you may have to bite the bullet and buy another MB if you're unable to bring Frankenstein back to life.http://www.amazon.com/gp/offer-listing/B00EVUW71A/ref=dp_olp_refurbished/184-1192468-3308525?ie=UTF8&condition=refurbished
The system damage has already been done and forcing the machine off and on is certainly not going to help your situation.It will not overheat as long as the fans are running and the cover is on. Let it do it's business, as interrupting a flash it will certainly Brick any machine.
Dell C1100: How to update BIOS and BMC to the latest version.
The sequence is not a secret....make sure you save an original copy of your ROM file.
If you "brick" your system, do the following:
(Dell C1100: How to update BIOS and BMC to the latest version.)So following this ^ post line for line actually ended up bricking my server when I was attempting to fix the "fan spinning up to 10,000 RPM every other second" issue. Reader beware if you have a "CS24-TY", do your research and figure out whether or not you actually have a Dell badged "Quanta".
Others with the same issue:
C1100 cloud server no POST after bios update (Link to THIS page you're on now.)http://en.community.dell.com/support-forums/cloud/f/4715/t/19492940.aspx
PowerEdge C1100 dead in the water... http://en.community.dell.com/techcenter/cloud/f/4442/t/19491173.aspx
Dell C1100 / CS24-TY bios and ems/ibmc update http://en.community.dell.com/techcenter/cloud/f/4442/t/19497619.aspx
Do you still have the Quanta BMC firmware available, or can you link to how to download it? Quanta has removed it from their website.