Attachment damages signature part 2

Question

I created code that adds an image to an existing pdf document and then signs it, all using PDFBox (see code below).

The code nicely adds the image and the signature. However, in some documents, Acrobat Reader complains that "The signature byte range is invalid."

The problem seems to be the same as the problem described in this question. The answer to that question describes the problem in more detail: the problem is that my code leaves a mix of cross reference types in the document (streams and tables). Indeed, some documents won't even open because of the problems that this creates.

My question is: how do I prevent this? How do I add an image to an existing pdf document without creating multiple cross reference types?

public class TC3 implements SignatureInterface{

private char[] pin = "123456".toCharArray();
private BouncyCastleProvider provider = new BouncyCastleProvider();
private PrivateKey privKey;
private Certificate[] cert;

public TC3() throws Exception{
    Security.addProvider(provider);
    KeyStore keystore = KeyStore.getInstance("PKCS12", provider);        
    keystore.load(new FileInputStream(new File("resources/IIS_keystore.pfx")), pin.clone());
    String alias = keystore.aliases().nextElement();
    privKey = (PrivateKey) keystore.getKey(alias, pin);
    cert = keystore.getCertificateChain(alias);
}

public void doSign() throws Exception{
    byte inputBytes[] = IOUtils.toByteArray(new FileInputStream("resources/rooster.pdf"));
    PDDocument pdDocument = PDDocument.load(new ByteArrayInputStream(inputBytes));
    PDJpeg ximage = new PDJpeg(pdDocument, ImageIO.read(new File("resources/logo.jpg")));
    PDPage page = (PDPage)pdDocument.getDocumentCatalog().getAllPages().get(0);
    PDPageContentStream contentStream = new PDPageContentStream(pdDocument, page, true, true);
    contentStream.drawXObject(ximage, 50, 50, 356, 40);
    contentStream.close();
    ByteArrayOutputStream os = new ByteArrayOutputStream();
    pdDocument.save(os);
    os.flush();        
    pdDocument.close();

    inputBytes = os.toByteArray(); 
    pdDocument = PDDocument.load(new ByteArrayInputStream(inputBytes));

    PDSignature signature = new PDSignature();
    signature.setFilter(PDSignature.FILTER_ADOBE_PPKLITE);
    signature.setSubFilter(PDSignature.SUBFILTER_ADBE_PKCS7_DETACHED);
    signature.setName("signer name");
    signature.setLocation("signer location");
    signature.setReason("reason for signature");
    signature.setSignDate(Calendar.getInstance());

    pdDocument.addSignature(signature, this);

    File outputDocument = new File("resources/signed.pdf");
    ByteArrayInputStream fis = new ByteArrayInputStream(inputBytes);
    FileOutputStream fos = new FileOutputStream(outputDocument);
    byte[] buffer = new byte[8 * 1024];
    int c;
    while ((c = fis.read(buffer)) != -1)
    {
        fos.write(buffer, 0, c);
    }
    fis.close();
    FileInputStream is = new FileInputStream(outputDocument);

    pdDocument.saveIncremental(is, fos);
    pdDocument.close();     
}

public byte[] sign(InputStream content) {
    CMSProcessableInputStream input = new CMSProcessableInputStream(content);
    CMSSignedDataGenerator gen = new CMSSignedDataGenerator();
    List<Certificate> certList = Arrays.asList(cert);
    CertStore certStore = null;
    try{
        certStore = CertStore.getInstance("Collection", new CollectionCertStoreParameters(certList), provider);
        gen.addSigner(privKey, (X509Certificate) certList.get(0), CMSSignedGenerator.DIGEST_SHA256);
        gen.addCertificatesAndCRLs(certStore);
        CMSSignedData signedData = gen.generate(input, false, provider);
        return signedData.getEncoded();
    }catch (Exception e){}
    return null;
}

public static void main(String[] args) throws Exception {
    new TC3().doSign();
}

Answer 1

The issue

As had already been explained in this answer, the issue at work here is that

• when non-incrementally storing the document with the added image, PDFBox 1.8.9 does so using a cross reference table no matter if the original file used a table or stream; if the original file used a stream, the cross reference stream dictionary entries are copied into the trailer dictionary;

  ...
  0000033667 00000 n
  0000033731 00000 n
  trailer
  <<
  /DecodeParms <<
  /Columns 4
  /Predictor 12
  >>
  /Filter /FlateDecode
  /ID [<5BD95916CAE5E84E9D964396022CBDCD> <6420B4547602C943AF37DD6C77496BE8>]
  /Info 6 0 R
  /Length 61
  /Root 1 0 R
  /Size 35
  /Type /XRef
  /W [1 2 1]
  /Index [20 22]
  >>
  startxref
  35917
  %%EOF
  

  (Most of these trailer entries here are useless or even misleading, see below.)

• when incrementally saving the signature, COSWriter.doWriteXRefInc uses COSDocument.isXRefStream to determine whether the existing document (the one we stored as above) uses a cross reference stream. As mentioned above, it does not. Unfortunately, though, COSDocument.isXRefStream in PDFBox 1.8.9 is implemented as

  public boolean isXRefStream()
  {
      if (trailer != null)
      {
          return COSName.XREF.equals(trailer.getItem(COSName.TYPE));
      }
      return false;
  }
  

  Thus, the misleading trailer entry Type shown above make PDFBox think it has to use a cross reference stream.

The result is a document whose initial revision ends with a cross reference table and weird trailer entries and whose second revision ends with a cross reference stream. This is not valid.

A work-around

Fortunately, though, understanding how the issue arises presents a work-around: Removing the troublesome trailer entry, e.g. like this:

    inputBytes = os.toByteArray();
    pdDocument = PDDocument.load(new ByteArrayInputStream(inputBytes));
    pdDocument.getDocument().getTrailer().removeItem(COSName.TYPE); // <<<<<<<<<< Remove misleading entry <<<<<<<<<<

With this work-around both revisions in the signed document use cross reference tables and the signature is valid.

Beware, if upcoming PDFBox versions change to save documents loaded from sources with cross reference streams using xref streams, too, the work-around must again be removed.

I would assume, though, that won't happen in the 1.x.x versions to come, and version 2.0.0 will introduce a fundamentally changed API, so the original code won't work out-of-the-box then anyhow.

---

Other ideas

I tried other ways, too, to circumvent this problem, trying to

• store the first manipulation as incremental update, too, or
• add the image during the same incremental update as the signature,

cf. SignLikeUnOriginalToo.java, but failed. PDFBox 1.8.9 incremental updates only seem to properly work for adding signatures.

---

Other ideas revisited

After looking into the creation of additional revisions using PDFBox some more, I tried the other ideas again and now succeeded!

The crucial part is to mark the added and changed objects as updated, including a path from the document catalog.

Applying the first idea (adding the image as an explicit intermediate revision) amounts to this change in doSign:

...
FileOutputStream fos = new FileOutputStream(intermediateDocument);
FileInputStream fis = new FileInputStream(intermediateDocument);

byte inputBytes[] = IOUtils.toByteArray(inputStream);

PDDocument pdDocument = PDDocument.load(new ByteArrayInputStream(inputBytes));
PDJpeg ximage = new PDJpeg(pdDocument, ImageIO.read(logoStream));
PDPage page = (PDPage) pdDocument.getDocumentCatalog().getAllPages().get(0);
PDPageContentStream contentStream = new PDPageContentStream(pdDocument, page, true, true);
contentStream.drawXObject(ximage, 50, 50, 356, 40);
contentStream.close();

pdDocument.getDocumentCatalog().getCOSObject().setNeedToBeUpdate(true);
pdDocument.getDocumentCatalog().getPages().getCOSObject().setNeedToBeUpdate(true);
page.getCOSObject().setNeedToBeUpdate(true);
page.getResources().getCOSObject().setNeedToBeUpdate(true);
page.getResources().getCOSDictionary().getDictionaryObject(COSName.XOBJECT).setNeedToBeUpdate(true);
ximage.getCOSObject().setNeedToBeUpdate(true);

fos.write(inputBytes);
pdDocument.saveIncremental(fis, fos);
pdDocument.close();

pdDocument = PDDocument.load(intermediateDocument);

PDSignature signature = new PDSignature();
...

(as in SignLikeUnOriginalToo.java method doSignTwoRevisions)

Applying the second idea (adding the image as part of the signing revision) amounts to this change in doSign:

...
byte inputBytes[] = IOUtils.toByteArray(inputStream);
PDDocument pdDocument = PDDocument.load(new ByteArrayInputStream(inputBytes));

PDJpeg ximage = new PDJpeg(pdDocument, ImageIO.read(logoStream));
PDPage page = (PDPage) pdDocument.getDocumentCatalog().getAllPages().get(0);
PDPageContentStream contentStream = new PDPageContentStream(pdDocument, page, true, true);
contentStream.drawXObject(ximage, 50, 50, 356, 40);
contentStream.close();

page.getResources().getCOSObject().setNeedToBeUpdate(true);
page.getResources().getCOSDictionary().getDictionaryObject(COSName.XOBJECT).setNeedToBeUpdate(true);
ximage.getCOSObject().setNeedToBeUpdate(true);

PDSignature signature = new PDSignature();
...

(as in SignLikeUnOriginalToo.java method doSignOneStep)

Both variants are clearly preferable to the original approach.