winzipaes is slow to decrypt a 10 MB file on Android

Question

I tried to decrypt a 10 MB file from a zip file with AES encryption on Samsung S5, but it's so slow, and that really surprise me. I'm familiar with AES, so I don't know if it consumes a lot of time. The following are my test results. Could anyone please tell me if these results are reasonable or not?

Is there anyway to speed up AES decryption?

PS. I use SpongyCastle to avoid classloader conflict, and I also modified winzipaes to use SpongyCastle.

Test 1
Device: Samsung S5
Zip archive: 7za a -tzip -mx=0 -p1234 -mem=AES256 test.zip 1MB_file 10MB_file
1MB_file: 1 MB
10MB_file: 10 MBs
test.zip: 12.5 MBs
compression rate: 1.00
Decrypting and uncompressing:
--> 1MB_file: 3167 ms
--> 10MB_file: 34137 ms

Test 2
Device: Samsung S5
Zip archive: 7za a -tzip -mx=1 -p1234 -mem=AES256 test.zip 1MB_file 10MB_file
1MB_file: 1 MB
10MB_file: 10 MBs
test.zip: 5.4 MBs
compression rate: 2.31
Decrypting and uncompressing:
--> 1MB_file: 1290 ms
--> 10MB_file: 15369 ms

Test 3
Device: Samsung S5
Zip archive: 7za a -tzip -mx=9 -p1234 -mem=AES256 test.zip 1MB_file 10MB_file
1MB_file: 1 MB
10MB_file: 10 MBs
test.zip: 5.1 MBs
compression rate: 2.46
Decrypting and uncompressing:
--> 1MB_file: 1202 ms
--> 10MB_file: 14460 ms

winzipaes: https://code.google.com/p/winzipaes/
SpongyCastle: http://rtyley.github.io/spongycastle/

==================================================================================

Using @Maarten Bodewes - owlstead solution

Test 2
Device: Samsung S5
Zip archive: 7za a -tzip -mx=1 -p1234 -mem=AES256 test.zip 1MB_file 10MB_file
1MB_file: 1 MB
10MB_file: 10 MBs
test.zip: 5.4 MBs
compression rate: 2.31
Decrypting and uncompressing:
--> 1MB_file: 206 ms (was 1290 ms)
--> 10MB_file: 1782 ms (was 15369 ms)

Answer 1

Yes, there are ways to speed this up as the source code of winzipaes uses a rather inefficient way of decrypting: it decrypt each block by calculating the IV and initializing the cipher (for CTR mode decryption). This could mean that the key gets reinitialized too often. Furthermore, handling data in blocks of 16 bytes is not very efficient either. This is mainly due to the fact that the AES-CTR as performed by WinZip uses a little endian counter instead of a big endian counter (as standardized).

The decryption also seems to include calculation of a HMAC-SHA1 over the ciphertext, which will add significant overhead as well. If you only require confidentiality of stored text then you may skip that step, although a MAC does have significant security advantages, providing cryptographically secure integrity as well as authenticity.

To show what I mean I've created a small sample code that at least runs a lot faster on my Java SE machine. According to Wayne (the original poster) this speeds up the Android code with a factor 10 or so, on my Java SE tests I "only" see speedups of about 3 times the original.

Changes:

• created special little endian counter mode used for ZIP
• simplified / optimized decrypter code because of above
• removed double key derivation (D'oh!) per file
• option not to validate MAC (with relatively small returns, SHA1 is pretty fast)
• using AESFastEngine, did not matter much, but hey...

It's very likely that the same kind of optimizations can be created for the encrypter.

Notes:

• .zip encryption is per stored file and therefore rather inefficient as the key derivation needs to take place once per stored file as well. Encryption of the .zip file itself would be much more efficient.
• using the JCA version of the decrypter may provide an a speedup as well as Android may be able to use OpenSSL code in the later versions (it will have to perform block by block encryption though).

-

/**
 * Adapter for bouncy castle crypto implementation (decryption).
 *
 * @author [email protected]
 * @author owlstead
 */
public class AESDecrypterOwlstead extends AESCryptoBase implements AESDecrypter {


    private final boolean verify;

    public AESDecrypterOwlstead(boolean verify) {
        this.verify = verify;
    }

    // TODO consider keySize (but: we probably need to adapt the key size for the zip file as well)
    public void init( String pwStr, int keySize, byte[] salt, byte[] pwVerification ) throws ZipException {
        byte[] pwBytes = pwStr.getBytes();

        super.saltBytes = salt;

        PBEParametersGenerator generator = new PKCS5S2ParametersGenerator();
        generator.init( pwBytes, salt, ITERATION_COUNT );

        cipherParameters = generator.generateDerivedParameters(KEY_SIZE_BIT*2 + 16);
        byte[] keyBytes = ((KeyParameter)cipherParameters).getKey();

        this.cryptoKeyBytes = new byte[ KEY_SIZE_BYTE ];
        System.arraycopy( keyBytes, 0, cryptoKeyBytes, 0, KEY_SIZE_BYTE );

        this.authenticationCodeBytes = new byte[ KEY_SIZE_BYTE ];
        System.arraycopy( keyBytes, KEY_SIZE_BYTE, authenticationCodeBytes, 0, KEY_SIZE_BYTE );

        // based on SALT + PASSWORD (password is probably correct)
        this.pwVerificationBytes = new byte[ 2 ];
        System.arraycopy( keyBytes, KEY_SIZE_BYTE*2, this.pwVerificationBytes, 0, 2 );

        if( !ByteArrayHelper.isEqual( this.pwVerificationBytes, pwVerification ) ) {
            throw new ZipException("wrong password - " + ByteArrayHelper.toString(this.pwVerificationBytes) + "/ " + ByteArrayHelper.toString(pwVerification));
        }

        cipherParameters = new KeyParameter(cryptoKeyBytes);

        // checksum added to the end of the encrypted data, update on each encryption call

        if (this.verify) {
            this.mac = new HMac( new SHA1Digest() );
            this.mac.init( new KeyParameter(authenticationCodeBytes) );
        }

        this.aesCipher = new SICZIPBlockCipher(new AESFastEngine());
        this.blockSize = aesCipher.getBlockSize();

        // incremented on each 16 byte block and used as encryption NONCE (ivBytes)

        // warning: non-CTR; little endian IV and starting with 1 instead of 0

        nonce = 1;

        byte[] ivBytes = ByteArrayHelper.toByteArray( nonce, 16 );

        ParametersWithIV ivParams = new ParametersWithIV(cipherParameters, ivBytes);
        aesCipher.init( false, ivParams );
    }

    // --------------------------------------------------------------------------

    protected CipherParameters cipherParameters;

    protected SICZIPBlockCipher aesCipher;

    protected HMac mac;


    @Override
    public void decrypt(byte[] in, int length) {
        if (verify) {
            mac.update(in, 0, length);
        }
        aesCipher.processBytes(in, 0, length, in, 0);
    }

    public byte[] getFinalAuthentication() {
        if (!verify) {
            return null;
        }
        byte[] macBytes = new byte[ mac.getMacSize() ];
        mac.doFinal( macBytes, 0 );
        byte[] macBytes10 = new byte[10];
        System.arraycopy( macBytes, 0, macBytes10, 0, 10 );
        return macBytes10;
    }
}

Of course you also need the referenced SICZIPCipher:

/**
 * Implements the Segmented Integer Counter (SIC) mode on top of a simple
 * block cipher. This mode is also known as CTR mode. This CTR mode
 * was altered to comply with the ZIP little endian counter and
 * different starting point.
 */
public class SICZIPBlockCipher
    extends StreamBlockCipher
    implements SkippingStreamCipher
{
    private final BlockCipher     cipher;
    private final int             blockSize;

    private byte[]          IV;
    private byte[]          counter;
    private byte[]          counterOut;
    private int             byteCount;

    /**
     * Basic constructor.
     *
     * @param c the block cipher to be used.
     */
    public SICZIPBlockCipher(BlockCipher c)
    {
        super(c);

        this.cipher = c;
        this.blockSize = cipher.getBlockSize();
        this.IV = new byte[blockSize];
        this.counter = new byte[blockSize];
        this.counterOut = new byte[blockSize];
        this.byteCount = 0;
    }

    public void init(
        boolean             forEncryption, //ignored by this CTR mode
        CipherParameters    params)
        throws IllegalArgumentException
    {
        if (params instanceof ParametersWithIV)
        {
            ParametersWithIV ivParam = (ParametersWithIV)params;
            byte[] iv = ivParam.getIV();
            System.arraycopy(iv, 0, IV, 0, IV.length);

            // if null it's an IV changed only.
            if (ivParam.getParameters() != null)
            {
                cipher.init(true, ivParam.getParameters());
            }

            reset();
        }
        else
        {
            throw new IllegalArgumentException("SICZIP mode requires ParametersWithIV");
        }
    }

    public String getAlgorithmName()
    {
        return cipher.getAlgorithmName() + "/SICZIP";
    }

    public int getBlockSize()
    {
        return cipher.getBlockSize();
    }

    public int processBlock(byte[] in, int inOff, byte[] out, int outOff)
          throws DataLengthException, IllegalStateException
    {
        processBytes(in, inOff, blockSize, out, outOff);

        return blockSize;
    }

    protected byte calculateByte(byte in)
          throws DataLengthException, IllegalStateException
    {
        if (byteCount == 0)
        {
            cipher.processBlock(counter, 0, counterOut, 0);

            return (byte)(counterOut[byteCount++] ^ in);
        }

        byte rv = (byte)(counterOut[byteCount++] ^ in);

        if (byteCount == counter.length)
        {
            byteCount = 0;

            incrementCounter();
        }

        return rv;
    }

    private void incrementCounter()
    {
        // increment counter by 1.
        for (int i = 0; i < counter.length && ++counter[i] == 0; i++)
        {
            ; // do nothing - pre-increment and test for 0 in counter does the job.
        }
    }

    private void decrementCounter()
    {
        // TODO test - owlstead too lazy to test

        if (counter[counter.length - 1] == 0)
        {
            boolean nonZero = false;

            for (int i = 0; i < counter.length; i++)
            {
                if (counter[i] != 0)
                {
                    nonZero = true;
                }
            }

            if (!nonZero)
            {
                throw new IllegalStateException("attempt to reduce counter past zero.");
            }
        }

        // decrement counter by 1.
        for (int i = 0; i < counter.length && --counter[i] == -1; i++)
        {
            ;
        }
    }

    private void adjustCounter(long n)
    {
        if (n >= 0)
        {
            long numBlocks = (n + byteCount) / blockSize;

            for (long i = 0; i != numBlocks; i++)
            {
                incrementCounter();
            }

            byteCount = (int)((n + byteCount) - (blockSize * numBlocks));
        }
        else
        {
            long numBlocks = (-n - byteCount) / blockSize;

            for (long i = 0; i != numBlocks; i++)
            {
                decrementCounter();
            }

            int gap = (int)(byteCount + n + (blockSize * numBlocks));

            if (gap >= 0)
            {
                byteCount = 0;
            }
            else
            {
                decrementCounter();
                byteCount =  blockSize + gap;
            }
        }
    }

    public void reset()
    {
        System.arraycopy(IV, 0, counter, 0, counter.length);
        cipher.reset();
        this.byteCount = 0;
    }

    public long skip(long numberOfBytes)
    {
        adjustCounter(numberOfBytes);

        cipher.processBlock(counter, 0, counterOut, 0);

        return numberOfBytes;
    }

    public long seekTo(long position)
    {
        reset();

        return skip(position);
    }

    public long getPosition()
    {
        byte[] res = new byte[IV.length];

        System.arraycopy(counter, 0, res, 0, res.length);

        for (int i = 0; i < res.length; i++)
        {
            int v = (res[i] - IV[i]);

            if (v < 0)
            {
               res[i + 1]--;
               v += 256;
            }

            res[i] = (byte)v;
        }

        // TODO still broken - owlstead too lazy to fix for zip
        return Pack.bigEndianToLong(res, res.length - 8) * blockSize + byteCount;
    }
}