Background:My company bought ~30 990s w/RAID1 setup over SED/AF drives (Seagate ST320LT014), vPro i5, no other customization.
Set up MDT2010 deployment from base W7SP1x64 image, latest drivers injected, selection profile to only 990 drivers (we have some Lenovos that use later chipset drivers). Followed the AF bulletin to the T - added the MSKB to the base image, using WINPE3.1, using Intel 10.x series drivers throughout. BIOS is upgraded to latest before the process starts.
Standard MDT task sequence modified to install DSS, then RST, then base programs (Office2K7, SEP, etc), then join domain. SCE installs after that.
After install, they work beautifully.
Until we starting installing new programs on them or giving them to people to work on. After a couple days the machine slows down to a slow drudge, with what looks like heavy disk contention issues. Takes 15minutes to start, 10 minutes to shut down, most programs unusable.
Windows, aside from the 100% activity level, shows no processes that are using the array in perfmon when the computer is idle. RST shows no rebuilding or verification in progress, reports normal.
I've tried sfc /scannow, chkdsk, manually verifying through RST, wiping and changing driver versions for chipset and SATA (including using Intel drivers for the Chipset), using a lower BIOS version. Dell diagnostics shows nothing out of the ordinary.
At this point I turned my attention to like hardware, finding some people with similar problems related to .Net - installing 4, updating, repairing the built in versions, none worked.
In fact, the only thing that stops the pain is shutting down, yanking a drive to break the array, and bringing the computer back up on one disk. RST has a fit about the degraded array, but the speed is back to normal. This makes me think that it may have something to do with the SED not being utilized (yet), or SED not being compatible with RAID1 while not being used?
Anyone have any ideas on further TS, or heard of this, or anything? I've done more troubleshooting than what I've written here, but I'm welcoming any and all ideas and am willing to retry most anything.
I remember some discussions about RAID configs when the AF drives started arriving, but not in relation to known issues. You have documented your issue thoroughly, so I would recommend that you contact Dell Tech Support to get them working on this issue for you.
Yeah, I'm in contact with them, but I'm having issues getting past the first level of support that asks me if I want to "format my hard drive" at my first mention of "advanced format" hard drives. And since the diagnostics come up negative, the third guy I talked to flat out said that it's obviously software and that's a fee-based support, then forcibly transferred me to that.
The Optiplex HW team I've gotten in touch with has been better, but I'm having communications issues with them thus far and am hoping to get someone that can do more than tell me to wipe and reload the machine.
This is really just a shot in the dark, as I've yet to see anyone online with my same exact problem after weeks of researching and hundreds of forums and posts.
Do you have an open support incident # that you can share?
I actually got in touch with someone that is better (Dennis, if you look into the notes - he's a good egg). He's having the motherboard replaced at request of his supervisor - which doesn't seem like the problem, unless I got a whole bad batch. If it's still broken after he's going to send me up to the next level people to talk about compatibility or driver issues. Not the best, but at least it's progress.
I'm going to send you the case# privately, in case you're still interested. It seems every time I call back I get a better tech, so maybe I'll just keep calling until I get someone that has seen this issue.
As an aside, I got the idea from a coworker to call up Seagate to see if they have any information on compatibility with my other hardware. The tech I talked to stated that while it technically should work in its current (non-encrypted) form, he couldn't be sure. But turning on the encryption would probably break the RAID and render the machine unbootable.
Therefore I've requested from my boss requisition of a non-SED set of disks to attempt to do the same install on, and hopefully he'll approve and talk to his Dell rep. It seems that if we can't encrypt the drives anyways, there's no sense in having them installed.
I am having a very similar problem for about two months. We are using XP x64bit on 990 MiniTowers with Advanced Format RAID 1 drives and it works perfect. When using the image on 990 Desktops with Advanced Format RAID 1 drives and after a couple of days the slowness shows up. The only cure is to shutdown, rebooting does not help. When the problem shows up the drive light stays on like the system is doing a lot of I/O. I can trigger this after a couple of days by starting a app that hasn't started in the last couple of days. Then it is over. I have updated every driver to the latest release, I've used Paragon Alignment Tool to make sure the new AF drives are without problems, they are. I have turned off every Power Option available, moved to OU's without any GPO's applied. Nothing has helped. I have a call open with Dell Tech Support, but nothing major has come of that, normal suggestions, update RST, BIOS, etc..
This only happens with our 990 Desktop Models, the 990 Mini Tower Models (still have the legacy format drives) do not have this problem. This is happening on the only 6 Desktop Models we have. This has me completely stumped
An additional note: Our Mini Tower models have Western Digital (WD25000BEKT-75PVMT0) drives that are fine. Our Desktop models have Seagate (ST250L0007-9ZV14C) drives that present the problems. I looked for a firmware upgrade for the Seagate drives, but one was not available.
I've got nothing for you, except Tech Support sent out a main board replacement last week, and today the problem still exists. Interesting that it's the Seagate drives only for you, or at least the form factor change.
I can't verify your finding of shutting down vs restart. I've done both and it seems that either will work, seemingly at random. I've had the problem go away on restart, and also the problem persist even over a shut down, unplug, and move to another location before boot - although that only happened once out of a few times, so small sample size there..
I'm calling them back now with the response now, and I'm hoping that I can get some better resolution from someone higher up.
I'm also installing non-AF/SED drives from my 780s into a 990 today and running tests to see if the same problem appears.
I'll make sure to keep this updated for your sake at the very least. It's an infuriating problem, I feel for you.
We appear to be having the same issue with our Optiplex 790's that are configured with the Seagate ST250LT0007-9ZV14C drives. We have several hundred 790's deployed with Toshiba and Western Digital drives that aren't having any issues. However, the units with the Seagate ST250LT0007-9ZV14C drives are behaving just as you described. Typically, they run fine for 3 to 6 days and then the Disk Response Time suddenly becomes extremely slow. As you mentioned, rebooting does not help, we need to shutdown the PC to resolve the issue. We're running Windows 7 64-bit on our systems instead of windows XP and we're not using RAID. I also have a case opened with Dell Tech Support.
Can you provide the Intel mass Storage driver (RST) version, BIOS version, and HDD part number (if you haven't already provided it) for the systems that are affected. The HDD team is looking into it, but has requested the extra details.
Hello, I have tried the following versions, but nothing had helped.
Intel Rapid Storage Technology -
Dell version - 10.1.0.1008, A07
Intel version - 126.96.36.1992
Bios - A07 (I believe this was the version when we received the systems), A10 and A11
HDD Part Number -
Dell - XDNFF or PPWYH, Manufacture - Seagate ST250LT007-9Z14C (Momentus Thin 250GB)
All other drivers (Video, NIC, etc.) have been updated to the latest as well, but nothing has helped. I have turned off all power options, moved the PCs into an OU with no GPOs applied, but nothing has helped. I have tried everything that I can think of, but after 3 or so days the systems just slow down and must be powered off. The Seagate drives were aligned, verified using Paragon Alignment Tool, RST shows the RAID to be in good shape. Nothing is captured in Event Viewer, Task Manager, Performance Monitor or Process Explorer. They just simply start to slow down.
RST version 10.1.0.1008
BIOS versions A07, A10, and A11
Dell HDD P/N: VV4P8
Drive Make / Model: Seagate ST250L0007-9ZV14C
I should have also mentioned that in addition to powering off the system, I've been able to resore the drive performance back to normal by putting the PC into Sleep mode and then waking it up and also by making a BIOS setting change. (In my case, I enabled SMART reporting, but I suspect any change would have had the same effect.)
When the issue is occuring for us, all of the performance metrics within Windows 7 look good except for the Drive Response Time. Typically, we see Disk Response times of 10 to 15 ms when starting an app such as IE or Word. However, when the drive is in this state, we're seeing response times that are anywhere between 1,500 to 20,000 ms.
Dell support sent me two new Toshiba drives for replacement. I installed them in RAID 1, as before about a week ago and applied my image. The system has run fine since. I just received 10 Toshiba drives for my 5 remaining systems.
One note: during my time of trouble shooting, I called Seagate tech support, the tech mentioned that this particular drive was not supported in any RAID configuration. I explained the systems were purchased from Dell with the RAID configuration. He said he could not support me as the drives now have OEM firmware on them. Kind of frustrating, but hopefully these posts help all that are reading.
Thanks for the follow-up information. I'll send it to the HDD team.
Final note to all - I achieved a resolution, of sorts. I was told that Seagate was involved and I was given the option to attempt firmware upgrades on our current drives. Instead, my boss negotiated replacement of all desktops to similar models with non SED/AF drives.
I won't be posting here further, as my role in this is finished. Good luck to all those that are in the same boat, hopefully Seagate's FW update will work.
We had a similar problem with the Seagate ST250LT007-9ZV14C drives in our Optiplex. The problem was resolved by installing the firmware update located here. We have been running smoothly ever since. Hope this helps!