Bouncy Castle DSA with SHA1 signature verification

Question

My project is using signature verification of some datasets which come from certain third-party software. Signature algorithm used is SHA1withDSA. When I was using standard SUN crypto provider, that comes with SDK, all went fine. Recently I switched to Bouncy Castle 1.50, and after that some of datasets which previously (that is, with SUN provider) stood verification, began to fail it, while the rest is still verified OK.

I explored source codes of both providers, and it turned out that SDK's default provider has some sort of protection from incorrectly formed signatures (while capable of being recovered), and Bouncy Castle provider does not have it. Check out OpenJDK for Java 7 (lines 336-344) or OpenJDK for Java 8 (lines 265-273): there they have made some signature fix in certain case. Whereas there is no such thing done for org.bouncycastle.jcajce.provider.asymmetric.dsa.DSASigner#engineVerify, moreover, in org.bouncycastle.crypto.signers.DSASigner#verifySignature it is explicitly stated that numbers must be positive, otherwise verification fails straight away.

Is it a bug in BC, or is there something that I missed? To overcome this, I have subclassed org.bouncycastle.crypto.signers.DSASigner and added there the same aforementioned signature fix, then plugged this in as yet another signature algorithm (through subclassing org.bouncycastle.jcajce.provider.asymmetric.dsa.DSASigner). But maybe there is another way that I overlooked, and this "issue" is well-known? Please advise.

Answer 1

If the incorrect BER/DER encoding of ASN.1 integers - which are stored as signed big endian, right aligned octets - is indeed the culprit then Bouncy does not have a bug. Positive values should be left padded with a 00 valued byte if the first bit of the encoding is set, otherwise it would represent a negative value.

The Sun provider is wrong to allow those kind of signatures to verify, and the other party is of course generating invalid signatures. Note that it is possible to let the signatures verify without this "fix" within the Sun code: simply adjust the encoding before feeding it to the verification function.

The only time when this is not possible is when the DSA verification is called as a generic signature verification method from another library instead of from an application that can adjust the data before the call.

On the other hand, I think you've created an elegant fix. The only issue with it is that it may not run if the provider's signature is verified from a JCA compliant framework. The other possible fix is to re-encode before feeding it into the Signature class for verification.

Note that I don't see how this could be a security issue; the signature consists of the values of R and S, and it does not matter how they are encoded, as long as you receive the correct values in the end.