Install a certificate for a local cluster

Question

I have some code to authenticate with Azure Key Vault in order to retrieve some secrets. I am authentication using a client id and certificate instead of a client id and secret. This code works great in a normal console app:

var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
try
{
    store.Open(OpenFlags.ReadOnly);

    var matchingCertificates = store.Certificates.Find(X509FindType.FindByThumbprint, thumbprint, false);
    if (matchingCertificates.Count != 1)
    {
        return null;
    }

    return matchingCertificates[0];
}
finally
{
    if (store != null) store.Close();
}

As soon as I try using this code in a stateful service application it is no longer able to find the certificate.

How can I install a certificate so that it is available to my local cluster?

Answer 1

The reason this doesn't work as-is is because Service Fabric runs under the NETWORK SERVICE account, which, without configuration, does not have access to the certificate.

For us at least, this scenario caused a "Keyset does not exist" exception when using the KeyVaultClient provided by Microsoft.Azure.KeyVault to connect to Key Vault using the certificate.

A solution to this is to include the certificate in your ApplicationManifest.xml, which instructs Service Fabric to give the NETWORK SERVICE account access to the certificate;

<ApplicationManifest>
   <!-- snip -->
   <Principals>
      <Users>
         <User Name="NetworkServiceAccount" AccountType="NetworkService" />
      </Users>
   </Principals>
   <Policies>
      <SecurityAccessPolicies>
         <SecurityAccessPolicy ResourceRef="KeyVaultCert" PrincipalRef="NetworkServiceAccount" GrantRights="Full" ResourceType="Certificate" />
      </SecurityAccessPolicies>
   </Policies>
   <Certificates>
      <SecretsCertificate X509FindValue="1ABCD86B815F37123459A34C1BA9EDEBABCEDF1" Name="KeyVaultCert" />
   </Certificates>
</ApplicationManifest>

... where both NetworkServiceAccount and KeyVaultCert are arbitrary names used by the <SecurityAccessPolicy /> element to reference the user and certificate.

See also https://azure.microsoft.com/en-us/documentation/articles/service-fabric-application-runas-security/#a-complete-application-manifest-example

Answer 2

Service Fabric applications run under the NETWORK SERVICE account, so you'll need to make sure that account has access/permissions to the certificate.

EDIT:

For a cluster running on your local box you do that by finding the certificate either using certmgr.msc or the relevant mmc snap-in and then right click > All Tasks > Manage Private Keys and then giving read permissions to NETWORK SERVICE.

For remote clusters in Azure, you can do that using a custom script extension on the VMs of the scale set that will run a PowerShell script that sets up the permissions you want. For example, it could do something like the following:

$certificate = Get-ChildItem -Path Cert:\LocalMachine\My | Where-Object {$_.Thumbprint -eq $certificateThumbprint}

# Get file path
$certificateFilePath = "C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys\" + $cert.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName

# Take ownership of the file so that permissions can be set
takeown /F $certificateFilePath

# Give the NETWORK SERVICE read permissions
$acl = (Get-Item $certificateFilePath).GetAccessControl('Access')
$rule = new-object System.Security.AccessControl.FileSystemAccessRule "NETWORK SERVICE","Read","Allow"
$acl.SetAccessRule($rule)
Set-Acl -Path $certificateFilePath -AclObject $acl