How to protect decryption key from decompilation?

Question

I'm a beginner java programmer. I'm working on an application that decrypts some data. The decryption key is hardcoded into the software and thus can be seen by analyzing the bytecode.

I know that reverse engineering cannot be prevented entirely so what I'm trying to do is to make the process as hard as possible.

My idea is not to directly put the key into my code but have it go through some kind of transformation. For example, I could write -

private static final byte[] HC256A = Hex
            .decode("8589075b0df3f6d82fc0c5425179b6a6"
                    + "3465f053f2891f808b24744e18480b72"
                    + "ec2792cdbf4dcfeb7769bf8dfa14aee4"
                    + "7b4c50e8eaf3a9c8f506016c81697e32");

This way someone looking at the bytecode can't read it straight away. But will have to follow the logic and apply transformations to it, which won't be that much easier at byte level.

So what do you guys think? Is this useful? What could be the be the best transformation other than hex decoding? Are there any other methods available to protect hardcoded decryption keys?

Thanks for all your suggestions.

Answer 1

Right way to attack such obfuscation (especially in bytecode languages) is to attach debugger to the place to which the key is passed (if debugging is not possible, start analyzing code from that place). This way the attacker doesn't need to look for the key at all and he doesn't care how obfuscated the key is. So you need to re-think your design.

If you only want to protect from the amateur lurkers, then splitting the key and XORing it's parts (possibly with different keys), would be enough. One more trick - derive the key from text constants already present in the code (such as application name). This makes the key less obvious than splitting or XORing.

Answer 2

Don't code the key into the source code at all. Keep it separate, ship it separately, e.g. in a Java keystore, and only to customers/sites/clients you trust, and put some legalese in the licence that places the onus on them if they leak the keystore.