Air-gapped .NET code-signed application will not install/run

Question

We recently updated our applications to make use of SHA-256 code-signing with a new certificate. The assemblies are strong name signed using the Sign the assembly option in Visual Studio 2015. The post build event in Visual Studio runs two signtool.exe processes to sign both in SHA-256 and for the legacy SHA-1 certificate:

call "C:\Program Files (x86)\Windows Kits\10\bin\x86\signtool.exe" 
sign /f "<mystrongName.pfx>" /p "<password>" /t 
<timestampURL> "$(TargetPath)"

call "C:\Program Files (x86)\Windows Kits\10\bin\x86\signtool.exe" 
sign /f "<mystrongName.pfx>" /p "<password>" /fd sha256 /tr 
<timestampURL> /td sha256 /as /v "$(TargetPath)"

Finally we use Advanced Installer as the installation packager and that too is code-signed on the Digital Signature page using the certificate and timestamp as per the .exe signature.

The final setup file installs and runs on Internet connected Windows machines as you would expect. You can see the certificate is assigned and valid, as well as the certificate chain through the properties of both the setup.exe and the runtime when installed. Furthermore, Windows recognizes the application as from a trusted source and displays the appropriate verified publisher details.

Our customer-base is largely global 100 companies and most of the deployments will be occurring in air-gapped networks. In one of our fist updated deployments in this environment, the certificate could not be verified preventing the installer from completing.

This made sense, because the Windows (2012 server R2) machines were isolated from the Internet and, due to company policies, had Turn off Automatic Root Certificates set to Enabled. This setting can be found in the Computer Configuration -> Administrative Templates -> System -> Internet Communication Management -> Internet Communication Settings folder of the MMC application (you need the certificates plugin installed).

When testing on our local test-bed, even machines not connected to the Internet would install the certificates from the setup utility if the above registry setting was the default (Disabled). We could replicate the issue by changing the policy setting to match the customers' (Enabled).

As a workaround, we manually downloaded the Certificate Authorities root certificate and installed it as a Trusted Root Certificate and the install would proceed normally.

When we presented this workaround to the customer, the installation still failed despite the Certificate Authorities root certificate being present in the Trusted Root Certificates of the machine.

The Certificate Authority customer service team recommended that we drop the timestamp from the signing process to allow the install to proceed - and that's the only help they offered (that's another story). However, this means that once the code-signing certificate expires, the application will either cease to run or will present unverified publisher errors.

I'm not totally convinced that this will fix the problem either, because when we tested locally the certificate was still found by the installer and allowed the installation to proceed when the Certificate Authorities root certificate was installed manually.

What I am unable to do is replicate the customers environment to exactly reproduce the problem (which doesn't help). It is almost as if Windows is bypassing the local machine's Trusted Root Certificates store. I am assuming that if this is possible it would be so that Windows can verify against a central root certificate store.

Is this even possible to set up in Windows? If so, where would I find either documentation on this or how is this done?

Am I missing something in the code-signing steps or in my understanding of what should be happening on the installing machine while it is checking the certificate?

I am at a loss as to what to do to get this installer working. What I can't afford to do is keep going back to the customer to get them to keep testing our installs. First-off it's really not the right process to debug, as the supplying vendor it isn't the customers problem to solve, but more importantly, I need our team to understand what is causing this and how to remedy it correctly.

Ideally I don't what to drop the timestamp if I don't have to because down the road this will cause new problems if the software doesn't get upgraded before the certificate expires.

Any and all help much appreciated.

Answer 1

I think one reason a certificate cannot be validated in an airgapped environment may be that revocation cannot be verified. As you may know, a certificate can be revoked, and there are two different protocols to check if it is, CRL and OCSP. Both require network access to the CA that issued the certificate.

Whether revocations are actually checked is governed by policies as described here, and this may cause your issues.