"Invalid provider type specified" CryptographicException when trying to load private key of certificate

Question

I'm trying to read the private key of a certificate which has been shared with me by a third-party service provider, so I can use it to encrypt some XML before sending it to them over the wire. I'm doing so programmatically in C#, but I think this is a permissions or misconfiguration issue, so I'll focus on the facts which seem to be most relevant:

• I don't think this issue is code-related; my code works on other computers, and the issue affects sample code from Microsoft.
• The certificate was provided as a PFX file and is just for test purposes, so it also includes a dummy certification authority.
• Using MMC.exe, I can import the certificate into the personal store for the local machine, before granting permissions on the private key to all relevant accounts, and dragging and dropping the certification authority into the Trusted Root Certification Authorities.
• Using C#, I can load the certificate (identified by its thumbprint) and verify that it has a private key using X509Certificate2.HasPrivateKey. However, trying to read the key causes an error. In .NET a CryptographicException is thrown with the message "Invalid provider type specified" upon trying to access the property X509Certificate2.PrivateKey. In Win32, calling the method CryptAcquireCertificatePrivateKey returns the equivalent HRESULT, NTE_BAD_PROV_TYPE.
• This is the same exception which also occurs when using two of Microsoft's own code samples to read the private key of the certificate.
• Installing the same certificate in the equivalent store for the current user, instead of the local machine, allows the private key to be successfully loaded.
• I'm on Windows 8.1 with local administrator rights, and I've tried running my code in both normal and elevated modes. Colleagues on Windows 7 and Windows 8 have been able to load the key from the local machine store for the same certificate.
• I can successfully read the private key of the self-signed IIS test certificate, which is in the same store location.
• I am already targeting .NET 4.5 (this error has been reported with some older versions of the framework).
• I don't think this is a problem with certificate templates, because I would expect that to affect both the local machine and current-user stores equally?

Unlike my colleagues, I have made multiple previous attempts to uninstall and re-install the certificate in various ways, including via IIS Manager and also including an older certificate from the same issuer. I can't see any traces of old or duplicate certificates in MMC. However, I do have many private key files of identical size which, based on the last-write time, must have been left behind after my various installation attempts. These are found at the following locations, for the local machine and current user stores respectively:

c:\ProgramData\Microsoft\Crypto\RSA\MachineKeys

c:\Users\\AppData\Roaming\Microsoft\Crypto\RSA\S-1-5-21-[rest of user ID]

So, can anyone please advise whether:

• It's a good idea to uninstall the certificate using MMC, delete all those files which look like orphaned private keys, and then re-install the certificate and try again?
• There any other files which I should try to manually delete?
• There's anything else I should try?

UPDATE - Added a code sample showing an attempt to read a private key:

static void Main()
{
    // Exception occurs when trying to read the private key after loading certificate from here:
    X509Store store = new X509Store("MY", StoreLocation.LocalMachine);
    // Exception does not occur if certificate was installed to, and loaded from, here:
    //X509Store store = new X509Store("MY", StoreLocation.CurrentUser);

    store.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);

    X509Certificate2Collection collection = (X509Certificate2Collection)store.Certificates;
    X509Certificate2Collection fcollection = (X509Certificate2Collection)collection.Find(X509FindType.FindByTimeValid, DateTime.Now, false);
    X509Certificate2Collection scollection = X509Certificate2UI.SelectFromCollection(fcollection, "Test Certificate Select", "Select a certificate from the following list to get information on that certificate", X509SelectionFlag.MultiSelection);
    Console.WriteLine("Number of certificates: {0}{1}", scollection.Count, Environment.NewLine);

    foreach (X509Certificate2 x509 in scollection)
    {
        try
        {
            Console.WriteLine("Private Key: {0}", x509.HasPrivateKey ? x509.PrivateKey.ToXmlString(false) : "[N/A]");
            x509.Reset();
        }
        catch (CryptographicException ex)
        {
            Console.WriteLine(ex.Message);
        }
    }
    store.Close();

    Console.ReadLine();
}

Answer 1

I had the same problem on Windows 8 and Server 2012/2012 R2 with two new certificates I recently received. On Windows 10, the problem no longer occurs (but that does not help me, as the code manipulating the certificate is used on a server). While the solution of Joe Strommen in principle works, the different private key model would require massive change to the code using the certificates. I find that a better solution is to convert the private key from CNG to RSA, as explained by Remy Blok here.

Remy uses OpenSSL and two older tools to accomplish the private key conversion, we wanted to automate it and developed an OpenSSL-only solution. Given MYCERT.pfx with private key password MYPWD in CNG format, these are the steps to get a new CONVERTED.pfx with private key in RSA format and same password:

1. Extract public keys, full certificate chain:

OpenSSL pkcs12 -in "MYCERT.pfx" -nokeys -out "MYCERT.cer" -passin "pass:MYPWD"

2. Extract private key:

OpenSSL pkcs12 -in "MYCERT.pfx" -nocerts -out "MYCERT.pem" -passin "pass:MYPWD" -passout "pass:MYPWD"

3. Convert private key to RSA format:

OpenSSL rsa -inform PEM -in "MYCERT.pem" -out "MYCERT.rsa" -passin "pass:MYPWD" -passout "pass:MYPWD"

4. Merge public keys with RSA private key to new PFX:

OpenSSL pkcs12 -export -in "MYCERT.cer" -inkey "MYCERT.rsa" -out "CONVERTED.pfx" -passin "pass:MYPWD" -passout "pass:MYPWD"

If you load the converted pfx or import it in the Windows certificate store instead of the CNG format pfx, the problem goes away and the C# code does not need to change.

One additional gotcha that I encountered when automating this: we use long generated passwords for the private key and the password may contain ". For the OpenSSL command line, " characters inside the password must be escaped as "".

Answer 2

In my case, I was trying to use a self-signed certificate with PowerShell's New-SelfSignedCertificate command. By default, it will generate a certificate using the CNG (Crypto-Next Generation) API instead of the older/classic crypto CAPI. Some older pieces of code will have trouble with this; in my case it was an older version of the IdentityServer STS provider.

By adding this at the end of my New-SelfSignedCertificate command, I got past the issue:

-KeySpec KeyExchange

Reference on the switch for the powershell command:

https://learn.microsoft.com/en-us/powershell/module/pkiclient/new-selfsignedcertificate?view=win10-ps

Answer 3

Here is another reason that this can happen, this was a weird issue and after struggling for a day I solved the issue. As an experiment I changed permission for "C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys" folder which holds private key data for certificates using Machine key store. When you change permission for this folder all privatekeys shows up as "Microsoft Software KSP provider" which is not the provider (in my case they are supposed to be " Microsoft RSA Schannel Cryptographic Provider").

Solution: Reset permissions to Machinekeys folder

Original permission for this folder can be found in here. In my case I have changed permission for "Everyone", gave read permissions where it removed "Special permissions" tick. So I checked with one of my team member (Right click folder> Properties > Security > Advanced > select "Everyone" > Edit > Click "Advanced settings" in permission check box list

Hope this will save someone's day!

Here is where I found the answer source, credit goes to him for documenting this.

Answer 4

In my case, the following code worked fine in localhost (both NET 3.5 and NET 4.7):

 var certificate = new X509Certificate2(certificateBytes, password);

 string xml = "....";
 XmlDocument xmlDocument = new XmlDocument();
 xmlDocument.PreserveWhitespace = true;
 xmlDocument.LoadXml(xml);

 SignedXml signedXml = new SignedXml(xmlDocument);
 signedXml.SigningKey = certificate.PrivateKey;

 //etc...

But it failed when deployed to an Azure Web App, at certificate.PrivateKey

It worked by changing the code as follows:

 var certificate = new X509Certificate2(certificateBytes, password, X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet | X509KeyStorageFlags.Exportable);
                                                                   //^ Here
 string xml = "....";
 XmlDocument xmlDocument = new XmlDocument();
 xmlDocument.PreserveWhitespace = true;
 xmlDocument.LoadXml(xml);

 SignedXml signedXml = new SignedXml(xmlDocument);
 signedXml.SigningKey = certificate.GetRSAPrivateKey();
                                      // ^ Here too

 //etc...

A whole day of work lost thanks to Microsoft Azure, once again in my life.

Answer 5

The link to Alejandro's blog is key.

I believe this is because the certificate is stored on your machine with the CNG ("Crypto Next-Generation") API. The old .NET API is not compatible with it, so it doesn't work.

You can use the Security.Cryptography wrapper for this API (available on Codeplex). This adds extension methods to X509Certificate/X509Certificate2, so your code will look something like:

using Security.Cryptography.X509Certificates; // Get extension methods

X509Certificate cert; // Populate from somewhere else...
if (cert.HasCngKey())
{
    var privateKey = cert.GetCngPrivateKey();
}
else
{
    var privateKey = cert.PrivateKey;
}

Unfortunately the object model for CNG private keys is quite a bit different. I'm not sure if you can export them to XML like in your original code sample...in my case I just needed to sign some data with the private key.

Answer 6

What worked for me: IIS / Application pool / Load User Profile = true