Encryption/decryption doesn't work well between two different openssl versions

Question

I've downloaded and compiled openssl-1.1.0.

I can encrypt and decrypt using the same exe of openssl (as is here)

me@ubuntu:~/openssl-1.1.0$ LD_LIBRARY_PATH=. ./apps/openssl aes-256-cbc -a -salt -in file.txt -out file.txt.enc enter aes-256-cbc encryption password: 123 Verifying - enter aes-256-cbc encryption password: me@ubuntu:~/openssl-1.1.0$ LD_LIBRARY_PATH=. apps/openssl aes-256-cbc -a -d -in file.txt.enc -out file.txt.dec enter aes-256-cbc decryption password: 123 

This openssl uses: libcrypto.so.1.1, libssl.so.1.1

When I try to decrypt with the openssl installed on my ubuntu, which uses: /lib/x86_64-linux-gnu/libssl.so.1.0.0, /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

I get an error:

me@ubuntu:~/openssl-1.1.0$ openssl aes-256-cbc -a -d -in file.txt.enc -out file.txt.dec2 enter aes-256-cbc decryption password: 123 bad decrypt 140456117421728:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:539: 

What may cause this? Thanks

Answer 1

The default digest was changed from MD5 to SHA256 in Openssl 1.1

Try using -md md5

cgs@ubuntu:~$ echo "it-works!" > file.txt cgs@ubuntu:~$ LD_LIBRARY_PATH=~/openssl-1.1.0/ openssl-1.1.0/apps/openssl aes-256-cbc -a -salt -in ~/file.txt -out ~/file.txt.enc -md md5 enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: cgs@ubuntu:~$ LD_LIBRARY_PATH=~/openssl-1.0.1f/ openssl-1.0.1f/apps/openssl aes-256-cbc -a -in ~/file.txt.enc -d enter aes-256-cbc decryption password: it-works! 

The ugly details:

The entered password is not used as is by aes (or other encryption) but the command implicitly derives a key from it. The key derivation uses message digest that was changed in openssl 1.1 Use SHA256 not MD5 as default digest.

In case you want to keep it simple password, and not start messing with the keying martial (-K,-iv) just force the same digest with -md

Answer 2

I tested the AES encryption and decryption with version 1.1.0a (downloaded from openssl.org) and the version 1.0.2g-fips (from my ubuntu 16.04)

When using the -p option on with 2 different versions of openssl, the IV and key are different:

$ LD_LIBRARY_PATH=~/openssl-1.1.0a/ ~/openssl-1.1.0a/apps/openssl aes-256-cbc -a -p -salt -in file -out file.enc enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: salt=6A80B2A3B4CFE048 key=637E17094DF7892A7AFC14957EAA13991DFFD3273A2459EDA613F3AD8A406C38 iv =6AC7CE5C9AADC6C46C633BF5124DAFBF  $ openssl aes-256-cbc -a -d -p -in file.enc -out file.dec enter aes-256-cbc decryption password: salt=6A80B2A3B4CFE048 key=6220AF2E25CB0B5D9994A0A1B05503D82AC5B0B4C9015E241CACBF8BF62DAC77 iv =2DC04EF29AA57478EBE606DF87277EA6 bad decrypt 140557073118872:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:592: 

I suspect a different derivation of key and IV based on the salt with the 2 versions.

If you want to get rid of this decryption error, you may remove the -salt option and use the options -K for the key and -iv in your openssl command.