Encrypting a large file with AES using JAVA

Question

I've tested my code with files less than this(10mb, 100mb, 500mb) and the encryption works. However, I run in to problems with files greater than 1gb. I've generated a large file (about 2gb) and I want to encrypt it with AES using JAVA, but I'm running into this error:

"Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

I've tried increasing available memory by using -Xmx8G, but no dice. Part of my code is as follows

    File selectedFile = new File("Z:\\dummy.txt");         
    Path path = Paths.get(selectedFile.getAbsolutePath());       
    byte[] toencrypt = Files.readAllBytes(path);       
    byte[] ciphertext = aesCipherForEncryption.doFinal(toencrypt);
    FileOutputStream fos = new FileOutputStream(selectedFile.getAbsolutePath());
    fos.write(ciphertext);
    fos.close();

As far as I can tell, the reason it is behaving this way, is that it is trying to read the whole file at once, encipher it, and store it into another byte array instead of buffering and streaming it in. Can anyone help me with some code tips?

I am a beginner to coding, so I don't really know much, any help will be appreciated.

Answer 1

Don't even try to read entire large files into memory. Encrypt a buffer at a time. Just do the standard copy loop with a suitably initialized CipherOutputStream wrapped around the FileOutputStream. You can use this for all files, no need to make a special case out of it. Use a buffer of 8k or more.

EDIT The 'standard copy loop' in Java is as follows:

byte[] buffer = new byte[8192];
int count;
while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

where in this case out = new CipherOutputStream(new FileOutputStream(selectedFile), cipher).

Answer 2

You can also simplify the process even further using Encryptor4j that I have authored: https://github.com/martinwithaar/Encryptor4j

File srcFile = new File("original.zip");
File destFile = new File("original.zip.encrypted");
String password = "mysupersecretpassword";
FileEncryptor fe = new FileEncryptor(password);
fe.encrypt(srcFile, destFile);

This library uses streaming encryption so it will not cause OutOfMemoryError even with large files. Also, instead of using passwords you can use your own Key as well.

Check out the example on the Github page here: https://github.com/martinwithaar/Encryptor4j#file-encryption

Answer 3

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;

import javax.crypto.Cipher;
import javax.crypto.spec.SecretKeySpec;

public class Cypher2021 {
    private static final String key = "You're an idiot!";
    private static final String ALGORITHM = "AES";
    private static final String TRANSFORMATION = "AES";

    public static void encrypt(File inputFile) {
        File encryptedFile = new File(inputFile.getAbsolutePath() + ".encrypted");
        encryptToNewFile(inputFile, encryptedFile);
        renameToOldFilename(inputFile, encryptedFile);
    }

    public static void decrypt(File inputFile) {
        File decryptedFile = new File(inputFile.getAbsolutePath() + ".decrypted");
        decryptToNewFile(inputFile, decryptedFile);
        renameToOldFilename(inputFile, decryptedFile);
    }

    private static void decryptToNewFile(File input, File output) {
        try (FileInputStream inputStream = new FileInputStream(input); FileOutputStream outputStream = new FileOutputStream(output)) {
            SecretKeySpec secretKey = new SecretKeySpec(key.getBytes(), ALGORITHM);
            Cipher cipher = Cipher.getInstance(TRANSFORMATION);
            cipher.init(Cipher.DECRYPT_MODE, secretKey);

            byte[] buff = new byte[1024];
            for (int readBytes = inputStream.read(buff); readBytes > -1; readBytes = inputStream.read(buff)) {
                outputStream.write(cipher.update(buff, 0, readBytes));
            }
            outputStream.write(cipher.doFinal());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static void encryptToNewFile(File inputFile, File outputFile) {
        try (FileInputStream inputStream = new FileInputStream(inputFile); FileOutputStream outputStream = new FileOutputStream(outputFile)) {
            SecretKeySpec secretKey = new SecretKeySpec(key.getBytes(), ALGORITHM);
            Cipher cipher = Cipher.getInstance(TRANSFORMATION);
            cipher.init(Cipher.ENCRYPT_MODE, secretKey);
            byte[] inputBytes = new byte[4096];
            for (int n = inputStream.read(inputBytes); n > 0; n = inputStream.read(inputBytes)) {
                byte[] outputBytes = cipher.update(inputBytes, 0, n);
                outputStream.write(outputBytes);
            }
            byte[] outputBytes = cipher.doFinal();
            outputStream.write(outputBytes);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static void renameToOldFilename(File oldFile, File newFile) {
        if (oldFile.exists()) {
            oldFile.delete();
        }
        newFile.renameTo(oldFile);
    }
}

And then you can use it like this:

import java.io.File;

public class Main {

    public static void main(String[] args) {
        File file = new File("text.txt");
        Cypher2021.encrypt(file); // converts "text.txt" into an encrypted file
        Cypher2021.decrypt(file); // converts "text.txt" into an decrypted file
    }
}