Best practices for passing encrypted data between different programming languages

Question

I have read that if you want to encrypt a string using one programming language and decrypt that string using another programming language, then to ensure compatibility it is best to do some conversions prior to doing the encryption. I have read that it's a best practice to encrypt the byte array of a string rather than the string itself. Also, I have read that certain encryption algorithms expect each encrypted packet to be a fixed length in size. If the last packet to be encrypted isn't the required size, then encryption would fail. Therefore it seems like a good idea to encrypt data that has first been converted into a fixed length, such as hex.

I am trying to identify best practices that are generally useful regardless of the encryption algorithm being used. To maximize compatibility when encrypting and decrypting data across different languages and platforms, I would like a critique on the following steps as a process:

Encryption:

• start with a plain text string
• convert plain text string to byte array
• convert byte array to hex
• encrypt hex to encrypted string
• end with an encrypted string

Decryption:

• start with an encrypted string
• decrypt encrypted string to hex
• convert hex to byte array
• convert byte array to plain text string
• end with a plain text string

Answer 1

Really the best practice for encryption is to use a high level encryption framework, there's a lot of things you can do wrong working with the primitives. And mfanto does a good a good job of mentioning important things you need to know if you don't use a high level encryption framework. And i'm guessing that if you are trying to maximize compatibility across programming languages, it's because you need other developers to inter-operate with the encryption, and then they need to learn the low level details of working with encryption too.

So my suggestion for high level framework is to use the Google Keyczar framework, as it handles the details of, algorithm, key management, padding, iv, authentication tag, wire format all for you. And it exists for many different programming Java, Python, C++, C# and Go. Check it out.

I wrote the C# version, so I can tell you the primitives it uses behind the scenes are widely available in most other programming languages too, and it uses standards like json for key management and storage.

Answer 2

Your premise is correct, but in some ways it's a little easier than that. Modern crypto algorithms are meant to be language agnostic, and provided you have identical inputs with identical keys, you should get identical results.

It's true that for most ciphers and some modes, data needs to be a fixed length. Converting to hex won't do it, because the data needs to end on fixed boundaries. With AES for example, if you want to encrypt 4 bytes, you'll need to pad it out to 16 bytes, which a hex representation wouldn't do. Fortunately that'll most likely happen within the crypto API you end up using, with one of the standard padding schemes. Since you didn't tag a language, here's a list of padding modes that the AesManaged class in .NET supports.

On the flip side, encrypting data properly requires a lot more than just byte encoding. You need to choose the correct mode of operation (CBC or CTR is preferred), and then provide some type of message integrity. Encryption alone doesn't protect against tampering with data. If you want to simplify things a bit, then look at a mode like GCM, which handles both confidentiality, and integrity.

Your scheme should then look something like:

• Convert plain text to string to byte array. See @rossum's comment for an important note about character encoding.
• Generate a random symmetric key or use PBKDF2 to convert a passphrase to a key
• Generate a random IV/nonce for use with GCM
• Encrypt the byte array and store it, along with the Authentication Tag
• You might optionally want to store the byte array as a Base64 string.

For decryption:

• If you stored the byte array as a Base64 string, convert back to the byte array.
• Decrypt encrypted byte array to plaintext
• Verify the resulting Authentication Tag matches the stored Authentication Tag
• Convert byte array to plain text string.