OpenSSL: can input and output buffers be the same for encrypt/decrypt routines?

Question

For example, in:

int EVP_EncryptUpdate(EVP_CIPHER_CTX *ctx, unsigned char *out,
                int *outl, unsigned char *in, int inl);

… can out == in?

Answer 1

I just stumbled upon this question because I was curious myself. Since no one answered, I tried it out and it does indeed work (at least with AES CTR 128 decryption) so I would venture to guess that it works for other types as well. Here's my code sample in case you are interested.

/* Test Vector from http://www.inconteam.com/software-development/41-encryption/55-aes-test-vectors */

const unsigned char key[16] = { 0x2b, 0x7e, 0x15, 0x16,
                                0x28, 0xae, 0xd2, 0xa6,
                                0xab, 0xf7, 0x15, 0x88,
                                0x09, 0xcf, 0x4f, 0x3c };

const unsigned char IV[16] = { 0xf0, 0xf1, 0xf2, 0xf3,
                               0xf4, 0xf5, 0xf6, 0xf7,
                               0xf8, 0xf9, 0xfa, 0xfb,
                               0xfc, 0xfd, 0xfe, 0xff };

unsigned char test[16] = { 0x6b, 0xc1, 0xbe, 0xe2,
                                 0x2e, 0x40, 0x9f, 0x96,
                                 0xe9, 0x3d, 0x7e, 0x11,
                                 0x73, 0x93, 0x17, 0x2a };

EVP_CIPHER_CTX mCtx;
EVP_DecryptInit(&mCtx, EVP_aes_128_ctr(), key, IV);
int out_size;
EVP_DecryptUpdate(&mCtx, test, &out_size, test, 16);