How to protect data protection key files with a certificate on Asp.Net Core 2 on debian/linux

Question

I'm trying to configuring data protection and to use the certificate to protect key files. Here is the MS documentation Configuring data protection

Here is what I'm trying to do:

services
    .AddDataProtection()
    .SetApplicationName("test server")
    .PersistKeysToFileSystem("/home/www-data/config")
    .ProtectKeysWithCertificate(
        new X509Certificate2("/home/www-data/config/"keyprotection.pfx);

When I launch the application I get the following error on startup:

info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
    Creating key {71e2c23f-448b-49c9-984f-3c8d7227c904} with 
    creation date 2017-08-29 18:53:51Z, activation date 2017-08-29 18:53:51Z, and expiration date 2017-11-27 18:53:51Z.
info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
    Writing data to file '/home/www-data/config/key-71e2c23f-448b-49c9-984f-3c8d7227c904.xml'.
fail: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[24]
    An exception occurred while processing the key element '<key id="71e2c23f-448b-49c9-984f-3c8d7227c904" version="1" />'.
System.Security.Cryptography.CryptographicException: Unable to retrieve the decryption key.
    at System.Security.Cryptography.Xml.EncryptedXml.GetDecryptionKey(EncryptedData encryptedData, String symmetricAlgorithmUri)
    at System.Security.Cryptography.Xml.EncryptedXml.DecryptDocument()
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.EncryptedXmlDecryptor.Decrypt(XElement encryptedElement)
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement)
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.DefaultKeyResolver[12]
    Key {71e2c23f-448b-49c9-984f-3c8d7227c904} is ineligible to be the default key because its CreateEncryptor method failed.
System.Security.Cryptography.CryptographicException: Unable to retrieve the decryption key.
    at System.Security.Cryptography.Xml.EncryptedXml.GetDecryptionKey(EncryptedData encryptedData, String symmetricAlgorithmUri)
    at System.Security.Cryptography.Xml.EncryptedXml.DecryptDocument()
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.EncryptedXmlDecryptor.Decrypt(XElement encryptedElement)
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.DeferredKey.<>c__DisplayClass1_0.<GetLazyDescriptorDelegate>b__0()
    at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
    at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
    at System.Lazy`1.CreateValue()
    at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyBase.get_Descriptor()
    at Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.CngGcmAuthenticatedEncryptorFactory.CreateEncryptorInstance(IKey key)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyBase.CreateEncryptor()
    at Microsoft.AspNetCore.DataProtection.KeyManagement.DefaultKeyResolver.CanCreateAuthenticatedEncryptor(IKey key)
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.DefaultKeyResolver[12]
    Key {71e2c23f-448b-49c9-984f-3c8d7227c904} is ineligible to be the default key because its CreateEncryptor method failed.
System.Security.Cryptography.CryptographicException: Unable to retrieve the decryption key.
    at System.Security.Cryptography.Xml.EncryptedXml.GetDecryptionKey(EncryptedData encryptedData, String symmetricAlgorithmUri)
    at System.Security.Cryptography.Xml.EncryptedXml.DecryptDocument()
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.EncryptedXmlDecryptor.Decrypt(XElement encryptedElement)
    at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement)
    at Microsoft.AspNetCore.DataProtection.KeyManagement.DeferredKey.<>c__DisplayClass1_0.<GetLazyDescriptorDelegate>b__0()
    at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location where exception was thrown ---

So the key is created and well encrypted. But it seems that somehow it doesn't know how to decrypt it as it says in the error:

System.Security.Cryptography.CryptographicException: 
    Unable to retrieve the decryption key.

If I understand it correctly, it uses the certificate I provided to encrypt the key. But it looks like it doesn't use the same cert for the decryption for some reason (It looks like it tries to retreive it from somewhere else [store?]).

What is going wrong ?

I also tried to put the cert into CA store as described here: Create a Self-Signed Certificate and trust it on Ubuntu Linux

Then I tried to find them back from the code like this:

var cert = new CertificateResolver().ResolveCertificate(CertThumbprint);

But it didn't work (it cannot find it).

I also tried tried to find them using the following approach:

var store = new X509Store(StoreName.CertificateAuthority,
    StoreLocation.LocalMachine);

store.Open(OpenFlags.ReadOnly);

var collection = store.Certificates.Find(
    X509FindType.FindByThumbprint,
    CertThumbprint, false);

store.Close();

var x509Cert = collection.Count > 0 ? collection[0] : null;

But it didn't work neither.

So what is the right way ?

Answer 1

For reasons known only to Microsoft, the ProtectKeysWithCertificate overrides that accept actual certificates (PFX files or X509Certificate2 objects) are only able to encrypt DPAPI data. Decryption only works if the same certificate is stored in the machine's certificate store, which makes those overrides relatively pointless.

Why? Who knows. It isn't particularly useful information, but it's vaguely dismissed here as a "limitation of the underlying framework".

In this related discussion (which was just closed without any Microsoft assistance or engagement at all), a user shares custom persistence classes which aren't affected this mysterious "limitation." GitHub repo linked below, I know this is an old question, but maybe it'll help someone else.

https://github.com/tillig/DataProtection

Update: This will be fixed in the upcoming Core 2.1.0 release: https://github.com/aspnet/Home/issues/2759#issuecomment-367157751