External signing PDF with iText

Question

First of all, although I've been following StackOverflow for quite some time, it is the first time I posted something, so if I do something wrong or not according to the rules, please feel free to point me in the right direction.

I'm developing a PDF digital signature application, using iText5, which depends on an external service to provide a signed hash after I prepare the PDF for signing.

As described in iText documentation, in the first phase I prepared the PDF (in the final implementation, all PDFs may be multi signed, so I use append mode), like so:

public static byte[] GetBytesToSign(string unsignedPdf, string tempPdf, string signatureFieldName, List<Org.BouncyCastle.X509.X509Certificate> certificateChain) {
        // we create a reader and a stamper
        using (PdfReader reader = new PdfReader(unsignedPdf)) {
            using (FileStream baos = File.OpenWrite(tempPdf)) {

                List<Org.BouncyCastle.X509.X509Certificate> chain = certificateChain;
                PdfStamper pdfStamper = PdfStamper.CreateSignature(reader, baos, '\0', null, true);
                sap                   = pdfStamper.SignatureAppearance;
                sap.Certificate       = certificateChain[0];
                sap.SetVisibleSignature(new iTextSharp.text.Rectangle(36, 720, 160, 780), 1, signatureFieldName);
                //sap.SetVisibleSignature(signatureFieldName);
                sap.SignDate          = DateTime.Now;
                PdfSignature dic      = new PdfSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);  
                dic.Date              = new PdfDate(sap.SignDate);
                dic.Name              = CertificateInfo.GetSubjectFields(chain[0]).GetField("CN");
                sap.CryptoDictionary  = dic;
                sap.Certificate       = certificateChain[0];
                sap.Acro6Layers       = true;
                sap.Reason            = "test";
                sap.Location          = "test";

                IExternalSignatureContainer external = new ExternalBlankSignatureContainer(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
                MakeSignature.SignExternalContainer(sap, external, 8192);
                signatureContainer = new PdfPKCS7(null, chain, "SHA256", false);
                byte[] hash = DigestAlgorithms.Digest(sap.GetRangeStream(), "SHA256");
                //byte[] signatureHash = signatureContainer.getAuthenticatedAttributeBytes(hash, null, null, CryptoStandard.CMS);

                return hash;
            }
        }
    }

After this step, I send the hash to the external service, which returns a signed hash.

Inspecting the hash that I send to the service, it seems to be correct, since it covers all the PDF except for the new signature contents.

I then conclude the signature process using the method below:

private byte[] Sign(PdfPKCS7 signatureContainer, List<X509Certificate2> chain2, List<Org.BouncyCastle.X509.X509Certificate> chain, byte[] hash, byte[] signedBytes, string tmpPdf, string signedPdf, string signatureFieldName) {
        System.Security.Cryptography.RSACryptoServiceProvider publicCertifiedRSACryptoServiceProvider = chain2[0].PublicKey.Key as System.Security.Cryptography.RSACryptoServiceProvider;
        bool verify = publicCertifiedRSACryptoServiceProvider.VerifyHash(hash, "SHA256", signedBytes); //verify if the computed hash is same as signed hash using the cert public key
        Console.WriteLine("PKey signed computed hash is equal to signed hash: " + verify);

        AsnEncodedData asnEncodedData = new AsnEncodedData(signedBytes);
        Console.WriteLine(asnEncodedData.Format(true));
        
        //ITEXT5
        try {
            //Console.WriteLine("Signed bytes: " + Encoding.UTF8.GetString(signedBytes));

            using (PdfReader reader = new PdfReader(tmpPdf)) {
                using (FileStream outputStream = File.OpenWrite(signedPdf)) {
                IExternalSignatureContainer external = new Objects.MyExternalSignatureContainer(signedBytes, chain, signatureContainer);
                MakeSignature.SignDeferred(reader, signatureFieldName, outputStream, external);
                }
            }
            return new byte[] { };
        }
        catch(Exception ex) {
            File.Delete(tmpPdf);
            Console.WriteLine("Error signing file: " + ex.Message);
            return new byte[] { };
        }
    }

In the beggining of the Sign method, I verify if the hash sent to the external service, signed with the same certificate, is equal to the external service response, which is true.

MyExternalSignatureContainer code:

public class MyExternalSignatureContainer : IExternalSignatureContainer {
        private readonly byte[] signedBytes;
        public List<Org.BouncyCastle.X509.X509Certificate> Chain;
        private PdfPKCS7 sigField;

        public MyExternalSignatureContainer(byte[] signedBytes) {
            this.signedBytes = signedBytes;
        }

        public MyExternalSignatureContainer(byte[] signedBytes, List<Org.BouncyCastle.X509.X509Certificate> chain, PdfPKCS7 pdfPKCS7) {
            this.signedBytes = signedBytes;
            this.Chain = chain;
            this.sigField = pdfPKCS7;
        }

        public byte[] Sign(Stream data) {
            try {
                sigField.SetExternalDigest(signedBytes, null, "RSA");
                return sigField.GetEncodedPKCS7(signedBytes, null, null, null, CryptoStandard.CMS);
            }
            catch (IOException ioe) {
                throw ioe;
            }
        }

        public void ModifySigningDictionary(PdfDictionary signDic) {
        }
    }

The problem is when I open the PDF in Acrobat, it states that the document has been modified or corrupted since the signature was applied.

(If I open the same PDF in PDF-XChange, it says the PDF wasn't modified).

What I've tried so far without luck:

Not being completely sure if the external service uses SHA256, I've already tried changing the digest to SHA1 of the pre-signing, resulting in a "Formatting error" in Acrobat Reader.

Like stated in another post in StackOverlow regarding the same issue (I'm unable to find the post to link it), a potential problem would be using different streams for the temporary file. I already tried to use the same stream without luck as well.

Samples of the PDF's:

Original file

Temp File

Signed File

Base64 hash sent to the service:

XYfaS/SisA/tk5hcl035RpBjOczrH9E5rgiAMpqgkjI=

Base64 signed hash sent in response:

CnV3WL7skhMCtZG1r1Qi2oyE9WPO3KP4Ieu/Xm4lec+DAbYbhQxCvjMISsG3sTwYY7Lqi4luD60uceViDH848rS9OkTn8szzAnnX2fSYIwqDpG3qjJAb6NOXEv41hy+XYhSBJWS4ji2mM2ReruwPafxB1aM25L5Jyd0V7WecuNFUevUrvd85Y2KBkyBw9zCA8NDAQPPY0UT4GkXZi3Z35+Sf/s2o8zxCOlBDaIJyMvJ9De79nw4jC5L9NesHpFxx3mX1g1N33GHjUNdETgFMhnd8RDUlGLW6bsAyv78gvwE6aXF6COObap/VtlLvMOME68MzLr6izKte6uA35Zwj9Q==

---

Update after mkl's answer:

According to the answer, I changed the code sign the document in only one phase and ended up with the following methods:

using (PdfReader reader = new PdfReader(fileLocation)) {
    using (FileStream baos = File.OpenWrite(tmpFile)) {

        List<Org.BouncyCastle.X509.X509Certificate> chain = Chain;
        PdfStamper pdfStamper = PdfStamper.CreateSignature(reader, baos, '\0', null, true);
        PdfSignatureAppearance sap = pdfStamper.SignatureAppearance;
        sap.Certificate = Chain[0];
        sap.SetVisibleSignature(new iTextSharp.text.Rectangle(36, 720, 160, 780), 1, signatureFieldName);
        //sap.SetVisibleSignature(signatureFieldName);
        sap.SignDate = DateTime.Now;
        PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
        dic.Date = new PdfDate(sap.SignDate);
        dic.Name = CertificateInfo.GetSubjectFields(chain[0]).GetField("CN");
        sap.CryptoDictionary = dic;
        sap.Certificate = Chain[0];
        sap.Acro6Layers = true;
        //sap.CertificationLevel = PdfSignatureAppearance.CERTIFIED_FORM_FILLING_AND_ANNOTATIONS;
        sap.Reason = "test";
        sap.Location = "test";

        IExternalSignature signature = new Objects.RemoteSignature(client, signatureRequest);
        MakeSignature.SignDetached(sap, signature, Chain, null, null, null, 8192, CryptoStandard.CMS);

    }
}

And the IExternalSignature implementation:

public virtual byte[] Sign(byte[] message) {
    IDigest messageDigest = DigestUtilities.GetDigest(GetHashAlgorithm());
    byte[] messageHash = DigestAlgorithms.Digest(messageDigest, message);
    //
    // Request signature for hash value messageHash
    // and return signature bytes
    //
    signatureRequest.Hash = messageHash;
    SignatureService.SignatureResponse signatureResponse = client.Signature(signatureRequest);

    if (signatureResponse.Status.Code == "00") {
         return signatureResponse.DocumentSignature;
    }
    else {
        throw new Exception("Error signing file: " + signatureResponse.Status.Message);
    }
}

The signatureResponse.DocumentSignature represents the signed bytes returned by the service.

In the result PDF now I'm a getting a BER decoding error.

Analyzing your example PDF you appear to declare the wrong certificate as signer certificate

Although I know the current certificate isn't valid, it is provided by the service and in the previous implementation of the service, where I would send the entire PDF for signing, the signed PDF was also signed with this certificate.

A question: Knowing that in a two-phase signing I was able to sign the PDF with this certificate (apart from the altered or corrupted document after signing error), shouldn't this method also work with the same certificate?

Currently, what is happening is this:

Inspecting the signature:

Again, if I open the same PDF in PDF-XChange, the signature is valid and the document was not modified. The requirement is the PDF to be valid in Acrobat, but I'm puzzled by this difference between readers.

Result PDF

---

Update 2

I.e. you only have to prefix your hash with the byte sequence 30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20.

After adding this SHA256 prefix to the message digest, the resulting PDF is now correctly signed.

Will Adobe Reader accept the fixed signature?

I doubt it. The key usage of the signer certificate only contains the value for signing other certificates.

The current certificate is only used for testing. In the production environment I believe the certificate provided by the external service will be valid.

I have two more questions regarding this issue:

For your code this means that you have to pack the hash into a DigestInfo structure before sending it to the service.

Q: How did you inspect the signature container to conclude that it wasn't correct?

Q: In my initial code, I had a two-phased signing. Would the same principal applied in a single-sign method still be valid, i.e., applying the SHA256 prefix do the pre-signed bytes and after setting the digest with the resulting signed bytes?

Answer 1

There are a number of issues in your code.

First of all your code mixes different iText signing API generations. There is the older API generation which requires you to work very near to the PDF internals, and there is the newer (since version 5.3.x) API which is implemented as a layer over the older API and does not require you to know those internals.

The "Digital Signatures for PDF documents" white paper focuses on showing the newer API, only section 4.3.3 "Signing a document on the server using a signature created on the client" uses the old API because the use case does not allow for the use of the newer API.

Your use case does allow for the use of the newer API, though, so you should try and use only it.

(In certain situations one can mix those APIs, but you then should really know what you're doing and still can get it awfully wrong...)

But now some more specific issues:

Working on closed objects

The MakeSignature.Sign* methods implicitly close the underlying PdfStamper and SignatureAppearance objects, so working with those objects thereafter should not be assumed to result in sensible information.

But in GetBytesToSign you do

MakeSignature.SignExternalContainer(sap, external, 8192);
signatureContainer = new PdfPKCS7(null, chain, "SHA256", false);
byte[] hash = DigestAlgorithms.Digest(sap.GetRangeStream(), "SHA256");

Thus, the sap.GetRangeStream() probably returns something wrong. (Probably it does still return the correct data but you shouldn't count on that.)

Signing the wrong bytes

GetBytesToSign returns the hash of the signed PDF document ranges:

signatureContainer = new PdfPKCS7(null, chain, "SHA256", false);
byte[] hash = DigestAlgorithms.Digest(sap.GetRangeStream(), "SHA256");
//byte[] signatureHash = signatureContainer.getAuthenticatedAttributeBytes(hash, null, null, CryptoStandard.CMS);

return hash;

Later, though, your code takes that return value, signs it, and tries to embed the returned signature bytes into the PdfPKCS7 signature container. This is wrong, the signature bytes have to be created for the authenticated attributes of the signer info of the signature container, not the document hash.

(By the way, here you use the older signature API without understanding it and, therefore, use it incorrectly.)

Putting the signed bytes in the wrong position

In MyExternalSignatureContainer you use the signed bytes in two calls:

sigField.SetExternalDigest(signedBytes, null, "RSA");
return sigField.GetEncodedPKCS7(signedBytes, null, null, null, CryptoStandard.CMS);

The first call is correct, here they belong. In the second call, though, the original hash of the signed document ranges should have been used.

(Here you again use the older signature API without understanding it and again use it incorrectly.)

###Supplying the wrong certificate

Analyzing your example PDF you appear to declare the wrong certificate as signer certificate. I think so because

• its public key cannot properly decrypt the signature bytes and
• the certificate is a CA certificate, not an end entity certificate, with inappropriate key usages for signing PDF documents.

How to improve your code

First of all, if I understand you correctly you request the signature from some other server, and that other server reacts quickly, so there is no need to free all resources while waiting for the signature. In such a situation there is no need for a two-phase signing process, you should do it in one step. All you need is a custom IExternalSignature implementation, something like

class RemoteSignature : IExternalSignature
{
    public virtual byte[] Sign(byte[] message) {
        IDigest messageDigest = DigestUtilities.GetDigest(GetHashAlgorithm());
        byte[] messageHash = DigestAlgorithms.Digest(messageDigest, message);
        //
        // Request signature for hash value messageHash
        // and return signature bytes
        //
        return CALL_YOUR_SERVICE_FOR_SIGNATURE_OF_HASH(messageHash);
    } 

    public virtual String GetHashAlgorithm() {
        return "SHA-256";
    } 

    public virtual String GetEncryptionAlgorithm() {
        return "RSA";
    } 
}

and use it like this for signing:

PdfReader reader = new PdfReader(...);
PdfStamper pdfStamper = PdfStamper.CreateSignature(...);
PdfSignatureAppearance sap = pdfStamper.SignatureAppearance;
// set sap properties for signing
IExternalSignature signature = new RemoteSignature();
MakeSignature.SignDetached(sap, signature, chain, null, null, null, 0, CryptoStandard.CMS);

Update of the IExternalSignature implementation

In the update of your question you added a PDF signed with the changes above applied. Analyzing the signature bytes in the signature container it became clear that your signing service is designed to be extremely dumb, it applies PKCS1 v1.5 padding and RSA encryption but it assumes its input to be already packed into a DigestInfo structure. In my experience this is an uncommon assumption, you should tell your signature provider to properly document that.

For your code this means that you have to pack the hash into a DigestInfo structure before sending it to the service.

A simple way to do this is explained in RFC 8017 section 9.2 note 1:

For the nine hash functions mentioned in Appendix B.1, the DER encoding T of the DigestInfo value is equal to the following:

    ...
    SHA-256: (0x)30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20 || H.
    ...

I.e. you only have to prefix your hash with the byte sequence 30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20.

Thus, a variant of the RemoteSignature class for services that require the caller to pack the digest into a DigestInfo structure could look like this:

class RemoteSignature : IExternalSignature
{
    public virtual byte[] Sign(byte[] message) {
        IDigest messageDigest = DigestUtilities.GetDigest(GetHashAlgorithm());
        byte[] messageHash = DigestAlgorithms.Digest(messageDigest, message);
        byte[] sha256Prefix = {0x30, 0x31, 0x30, 0x0d, 0x06, 0x09, 0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01, 0x05, 0x00, 0x04, 0x20};
        byte[] digestInfo = new byte[sha256Prefix.Length + messageHash.Length];
        sha256Prefix.CopyTo(digestInfo, 0);
        messageHash.CopyTo(digestInfo, sha256Prefix.Length);
        //
        // Request signature for DigestInfo value digestInfo
        // and return signature bytes
        //
        return CALL_YOUR_SERVICE_FOR_SIGNATURE_OF_DIGEST_INFO(digestInfo);
    } 

    public virtual String GetHashAlgorithm() {
        return "SHA-256";
    } 

    public virtual String GetEncryptionAlgorithm() {
        return "RSA";
    } 
}

Will Adobe Reader accept the fixed signature?

I doubt it. The key usage of the signer certificate only contains the value for signing other certificates.

If you look into the Adobe Digital Signatures Guide for IT, you'll see that valid key usage extensions are

• absent, i.e. no key usage extension at all, or
• present with one or more of the following values:
  • nonRepudiation
  • signTransaction (11.0.09 only)
  • digitalSignature (11.0.10 and later)

Thus, the signCertificate value of your certificate might be a problem.