PEM_read_bio_PrivateKey() returns NULL in ECB mode only

Question

PEM_read_bio_PrivateKey() returns NULL if a private key is encrypted by DES EDE in ECB mode. The issue happens in EVP_DecryptFinal_ex():

4128:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:330:

If the same private key is encrypted by DES EDE in CBC mode, this function works OK.

I checked, this issue is reproducible on openssl 0.9.8r/y versions (without FIPS). If openssl is built with FIPS the issue doesn't happen.

What causes this behaviour?

Thanks!

Answer 1

Looking at OpenSSL sources, this can happen only if padding in encrypted data is invalid. Does this happen with different keys, and can you provide a sample one? And, what tool is used to generate such keys (i.e. the same OpenSSL or not)? Probably, it omits padding/or writes zero padding instead of correct one.