Today we became aware that a certificate (eDellRoot), installed by our Dell Foundation Services application on our PCs, unintentionally introduced a security vulnerability. The certificate was implemented as part of a support tool and intended to make it faster and easier for our customers to service their system. Customer security and privacy is a top concern and priority for Dell; we deeply regret that this has happened and are taking steps to address it.
The certificate is not malware or adware. Rather, it was intended to provide the system service tag to Dell online support allowing us to quickly identify the computer model, making it easier and faster to service our customers. This certificate is not being used to collect personal customer information. It’s also important to note that the certificate will not reinstall itself once it is properly removed using the recommended Dell process.
We have posted instructions to permanently remove the certificate from your system here. We will also push a software update starting on November 24 that will check for the certificate, and if detected remove it. Commercial customers who reimaged their systems without Dell Foundation Services are not affected by this issue. Additionally, the certificate will be removed from all Dell systems moving forward.
Your trust is important to us and we are actively working to address this issue. We thank customers such as Hanno Böck, Joe Nord and Kevin Hicks, aka rotorcowboy, who brought this to our attention. If you ever find a potential security vulnerability in any Dell product or software, we encourage you to visit this site to contact us immediately.
*****UPDATE 11/25/2015*****
Since Monday, our teams have been working hard to address the security issue caused by the eDellRoot certificate. When we became aware of the issue, we immediately dug into all our applications that get pre-loaded on our PCs. We can confirm we have found no other root certificates on our factory installed PC images. What we did find was that the Dell System Detect application and its DSDTestProvider root certificate had similar characteristics to eDellRoot. Thank you again Hanno Böck for calling this to our attention, as well as topg who commented below.
In the case of Dell System Detect, our customer opts to download the software proactively to interact with our support website so we can provide a better and more personalized experience. Like eDellRoot, the certificate in question was designed to make it faster and easier for our customers to get support. Unlike eDellRoot, this certificate is not related to software that was pre-installed on our systems.
The impact from Dell System Detect is limited to customers who used the “detect product” functionality on our support site between October 20 and November 24, 2015. The application in question was removed from the support site on November 24 and a replacement application without the certificate is now available. We are proactively pushing a software update to address the issue for our consumer systems. Our commercial customers can either manually remove the certification or use their system management tools like SCCM to do so (we will be providing instructions on this shortly). If you choose the manual option, we have updated instructions on our site http://www.dell.com/support/edellroot to permanently remove this certificate. Note, these are updated instructions for removing both eDellRoot and DSDTestProvider from any folders where they may be stored. If you previously uninstalled eDellRoot, we recommend you go through the process again to ensure a thorough sweep.
WIRED has noted that “security is far easier to promise than it is to achieve.” We know that your trust is harder to win than it is to lose. Once we know we have addressed these issues and our customers have what they need to ensure their systems are safe, we will provide an account of how the issues were introduced – not only for your information, but so we can improve our processes.
In today’s world of ever-increasing cybersecurity threats, we all need to be vigilant. And that is the promise that we make - Dell will remain ever vigilant against security threats and we will respond with the utmost speed and accuracy when we become aware of issues that can impact our customers.
*****UPDATE 11/30/2015*****
Today Microsoft released Security Advisory 3119884 that will place both the eDellRoot and DSDTestProvider certificates into the Windows Certified Trust List (CTL) as non-trusted certificates, so even if the certificates are installed, they cannot be used. CTL updates are automatically pushed to both consumer and commercial Windows PCs. Most systems with Internet access should pick up the update within the next 24 hours. For more information, see the Security Advisory. This security step is in addition to actions already taken by Dell, as outlined in this post early last week, and by partners like Microsoft and Intel who added the patch to their anti-virus, anti-malware tools on November 25 to ensure both certificates were no longer usable.
Jeff Clarke, our vice chairman and president of Client Solutions, came into the studio to tape a short message to our customers and the security community to underscore our commitment to your security, to getting these issues resolved, and to being forthcoming with information as we have it.
With this latest important step by Microsoft and the proactive security updates driven by Dell complete, we are now turning our full attention to understanding what happened and how to prevent it in the future. We will be sure to update you here when we have more information to share.
To post a comment login or create an account
Unrelated comments or requests for service will be unpublished. Please post your technical questions in the Support Forums or for direct assistance contact Dell Customer Service or Dell Technical Support.. All comments must adhere to the Dell Community Terms of Use.
What about the equally problematic DSDTestProvider root certificate that seems to have been installed by Dell System Detect on my XPS 13? It has the same properties as eDellRoot & also includes a private key ...
:D
What this indicates is that the certificate was generated outside an HSM; or was exported from HSM or was exported from it. Either would be very bad certificate management practice.
I read about this vulnerability in an article on BBC online. Has not hit the US news?
Big issue: no where (even in the linked instructions) does it say which PCs are affected? The instructions imply that only Windows 8.1/10 machines are affected? Desktops and laptops?
Is there a silent switch for the automatic removal tool method? We would like to be able to push it out via SCCM.
For example: eDellRootCertFix.exe /quiet
For the sake of the rest of the industry, please publish an account of why this happened - the security world would like to understand the failures in process and thought that led to the creation of a relatively complex use of cryptographic components in a way that would ring alarm bells for most practiced cryptography users. If it "just seemed like a good idea" to some developer who didn't speak to a security professional, or it was reviewed by a security team, those are different problems, and which one it was will add to the corpus of knowledge used by application security professionals to improve security involvement in development process. Don't let this languish in the realm of "oops, we did a bad thing, and now we made it better".
You have a responsibility to tell us what the bad thing was, and what steps you did to make it better - otherwise, the only responsible conclusion to draw is that you're idiots, you employ idiots, and you'll continue to employ idiots who do idiotic things.
I'm fairly certain that's not the case, but I think the security community needs you to explain why that's not true.
@ameyer117 Try this command for SCCM deployment:
certutil.exe -delstore root "6b c5 7b 95 18 93 aa 97 4b 62 4a c0 88 fc 3b b6"
That's the eDellRoot serial number.
after running the .exe I've noticed that eDellRoot still exist in the personal certificate store in local computer certificates. Why does it not delete the certificate?
Also we'd like to deploy it via GPO, any chance of a .MSI?
I ran the eDellRootCertFix.exe on my new (just last week) Dell OptiPlex 7020 running Win10. Now I can't access my start menu. I get a big green box that says "Critical Error. Your start menu isn't working. We'll try to fix it the next time you sign in." Logging out and back in again does not fix the problem. Rebooting my computer does not fix the problem. Dell's RootCertFix program has made my computer inoperable. Any suggestions on how to make my computer work again?
This is one of those reasons I always uninstall anything with "Dell" in the name from newly acquired systems, including "Dell Foundation Services". Unfortunately, I don't know whether simply uninstalling Dell Foundation Services actually removes the offending certificate.
The "fix-tool" released today appears to solve the problem, but for those of us that have to manage hundreds or even thousands of systems, it is "noisy" popping up "you're not affected" or "you've been fixed" boxes which are intrusive to the user. So we can't script a rollout of that tool.
Can we get a silent version that simply writes log results to a file so IT administrators can simply retrieve the file to interpret the success/failure results? Or even just a silent version? This would be a 3-minute issue for me to fix by rolling out to the 100+ companies we support if it weren't for the non-silent nature of the current fix tool.
@DanielHarsh
That works, except the Word doc instructions Dell provides shows more steps. Specifically, when the 'Dell Foundation Services' service first starts, it re-installs the cert. So, first, one has to stop that service and delete the Dell.Foundation.Agent.Plugins.eDell.dll" file (or uninstall the software completely).
I use XP SP3 on a Dell Vostro 200 PC. Can I assume this does not apply to me?
@Louis D - Any commercial and consumer systems that received an update to Dell Foundation Services beginning in August 2015 were impacted. This update was removed on 11/23 and was replaced by a new update that will eliminate the root certificate from systems. Commercial customers who reimaged their systems without the Dell Foundation Services application were not impacted.
@gerben_z - Sorry to hear of the issue. For technical assistance contact Dell Customer Service or Dell Technical Support (links above under "Content Reminder"), or if you are on Twitter you can reach out to @DellCaresPro for assistance.
@topg - Wanted to let you know that our team is looking into this, as well, and we should have an update to address it very soon.
> "Today we became aware that a certificate (eDellRoot), installed by our Dell Foundation Services application on our PCs, unintentionally introduced a security vulnerability."
Dell is selling equipment to customer accounts with full knowledge the customers have SAS 70, SSAE 16, SOC or HIPAA certification requirements. Dell also has the ability to set the policies of SonicWall, EMC and VMware. The amount of confidential information, both enterprise and personal, that are impacted by Dell's policies is staggering. Therefore, it is extremely alarming that Dell is claiming incompetence is the reason for installing a rogue root certificate authority with a publically available private key. If Dell has reached the level of incompetence that they truly do not understand the basic concepts of the impact the root certificate authority entry has on security then how can Dell be trusted to actually be following a security criteria required by its customers?
Why doesn't Dell have a responsible computer security researcher on staff to monitor and sign off on change to the OS security layers? What exactly does Dell SecureWorks do and what hope is there of them having positive results for customers if their own house is in a state of such disarray?
At this point, Dell might want to consider paying Bruce Schneier or Daniel Bernstein whatever it takes to get them to join the company because I think that is the extent that Dell will need to go to recover its image.
> "This certificate is not being used to collect personal customer information."
FALSE FALSE FALSE FALSE FALSE!!!!
This statement further illustrates the degree to which Dell does not understand the impact it has.
The private key for this certificate is available publicly. Dell does not have complete control over over the *use* of this certificate. Instead, any third party can leverage this certificate to perform a man in the middle attack to view traffic from Dell customers. Once the third party takes advantage of the vulnerability against Dell customers, it will be "this certificate" which is used to assist in the collection of personal customer information including the possibility of collecting information related to online banking and healthcare. The fact that Dell itself isn't using the certificate to collect personal customer information does make your statement any less false.
If Dell really understood the magnitude of liability Dell has created for itself and for its customers, a warning would be issued on Dell's primary web page instead of buried in a community forum. Dell's handling of this issue, both on twitter and here, has been extremely upsetting given the amount of trust people have placed in Dell.
@Ivorygate
Whoops! I missed that. That's crazy that it reinstalls constantly. Well - nevermind my seemingly temporary fix then; that's why I wanted someone to test it. Cheers :)
My Services does not contain an entry for "Dell Foundation Services," and my directory c:\Program Files\Dell\ does not contain the subdirectory "Dell Foundation Services." Does that mean I don't need this fix?
@Laura P. Thomas
Can you clarify?
I haven't received the software update which was stated as being released on November 24? I have clicked several times on check for updates but nothing?
This is only a problem for those running Windows OS. If you are running only Linux then it is not a problem.
This is not a complete fix. If you have to reinstall your OS, it will also reinstall infected certificates. Additionally, the certificates were found in the "User Certificates" and "Computer Certificates." Your fix needs work.
Powershell Script, set it as a computer startup script. We have already made the move to Windows 10 but this should work from Windows 7 onwards.
Stop-Service -Name 'Dell Foundation Services'
Set-Service -Name 'Dell Foundation Services' -StartupType Disabled
Remove-Item 'C:\Program Files\Dell\Dell Foundation Services\Dell.Foundation.Agent.Plugins.eDell.dll' -Force
I have read that this affects newer laptops and desktops. Can you be more specific as to what is meant by "newer", and what systems this affects? Also, does this affect servers?
Our organization has many dell servers, desktops, and laptops and it would be very cumbersome to check every single one
I ran the removal "eDellRootCertFix.exe" command from the Dos Prompt with different commands /silent or /quiet and there still a pop up message. How can I deploy this solution silently? This is a big issue that needs a WAN solution. Please help!!!