IllegalBlockSizeException "null" in RSA decryption on Android

Question

I'm currently working on an Android client of my encryption software, but I kept getting IllegalBlockSizeException, and e.getMessage() always returns null

Here's the code I used to find the problem

try {
    KeyPairGenerator generator = KeyPairGenerator.getInstance(KeyProperties.KEY_ALGORITHM_RSA,"AndroidKeyStore");
    generator.initialize(new KeyGenParameterSpec.Builder(
            "1",
            KeyProperties.PURPOSE_ENCRYPT | KeyProperties.PURPOSE_DECRYPT)
            .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_OAEP)
            .setDigests(KeyProperties.DIGEST_SHA256)
            .setKeySize(2048)
            .build());
    KeyPair kp = generator.generateKeyPair();

    KeyStore store = KeyStore.getInstance("AndroidKeyStore");
    store.load(null);
    PublicKey pubKey = store.getCertificate("1").getPublicKey();
    PrivateKey privkey = ((KeyStore.PrivateKeyEntry) store.getEntry("1",null)).getPrivateKey();

    byte[] content = "123456789".getBytes();
    Cipher encrypt = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    encrypt.init(Cipher.ENCRYPT_MODE,pubKey);
    Cipher decrypt = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    decrypt.init(Cipher.DECRYPT_MODE,privkey);

    byte[] A = encrypt.doFinal(content);
    byte[] B = decrypt.doFinal(A);

    String resultA = new String(A);
    String resultB = new String(B);
    Toast.makeText(getApplicationContext(), resultA + "\n" + resultB, Toast.LENGTH_LONG).show();
} catch (Exception e) {
    e.printStackTrace();
    Toast.makeText(getApplicationContext(), e.toString()+"\n"+e.getMessage(), Toast.LENGTH_LONG).show();
}

As I said there was an IllegalBlockSizeException, which I found was thrown by decrypt.doFinal() , and e.getMessage() returns null

Here's what I get from the debug console

W/System.err: javax.crypto.IllegalBlockSizeException
    at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:519)
    at javax.crypto.Cipher.doFinal(Cipher.java:1741)
W/System.err:     at storm.cyanine.decryptor.MainActivity$override.onJob(MainActivity.java:187)
    at storm.cyanine.decryptor.MainActivity$override.onOpen(MainActivity.java:115)
    at storm.cyanine.decryptor.MainActivity$override.access$dispatch(Unknown Source:50)
    at storm.cyanine.decryptor.MainActivity.onOpen(Unknown Source:15)
    at java.lang.reflect.Method.invoke(Native Method)
    at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.onClick(AppCompatViewInflater.java:385)
W/System.err:     at android.view.View.performClick(View.java:6329)
    at android.view.View$PerformClick.run(View.java:24996)
    at android.os.Handler.handleCallback(Handler.java:809)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:166)
    at android.app.ActivityThread.main(ActivityThread.java:7377)
W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:469)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:963)
W/System.err: Caused by: android.security.KeyStoreException: Unknown error
    at android.security.KeyStore.getKeyStoreException(KeyStore.java:709)
    at android.security.keystore.KeyStoreCryptoOperationChunkedStreamer.doFinal(KeyStoreCryptoOperationChunkedStreamer.java:224)
    at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:506)
    ... 16 more

At first, I thought it was some restriction of AndroidKeyStore that prevents me from using the private key, so I tried using directly the one generated by KeyPairGenerator but nothing changed

What could cause this problem?

I've been searching online for days and I found nothing wrong with the code above (apart from not being the final product)

My device is Android 8.1.0

Edit: Thanks Steve Miskovetz a lot for the solution, and I just found the Android official guide for this topic: Cryptography

(don't know why I wasn't able to find this earlier)

Answer 1

It looks like your issue was introduced with Android Oreo, but has a workaround.

This stackoverflow post discusses it:
Android 8.0: IllegalBlocksizeException when using RSA/ECB/OAEPWithSHA-512AndMGF1Padding

This Google issue tracker has good discussion on it:
https://issuetracker.google.com/issues/36708951

You need to add this line:

OAEPParameterSpec sp = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-1"), PSource.PSpecified.DEFAULT);

And modify these lines, adding the sp parameter:

encrypt.init(Cipher.ENCRYPT_MODE,pubKey,sp);
decrypt.init(Cipher.DECRYPT_MODE,privkey,sp);

Your full code with modifications here:

try {
    KeyPairGenerator generator = KeyPairGenerator.getInstance(KeyProperties.KEY_ALGORITHM_RSA,"AndroidKeyStore");
    generator.initialize(new KeyGenParameterSpec.Builder(
            "1",
            KeyProperties.PURPOSE_ENCRYPT | KeyProperties.PURPOSE_DECRYPT)
            .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_OAEP)
            .setDigests(KeyProperties.DIGEST_SHA256)
            .setKeySize(2048)
            .build());
    KeyPair kp = generator.generateKeyPair();

    KeyStore store = KeyStore.getInstance("AndroidKeyStore");
    store.load(null);
    PublicKey pubKey = store.getCertificate("1").getPublicKey();
    PrivateKey privkey = ((KeyStore.PrivateKeyEntry) store.getEntry("1",null)).getPrivateKey();

    byte[] content = "123456789".getBytes();
    OAEPParameterSpec sp = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-1"), PSource.PSpecified.DEFAULT);
    Cipher encrypt = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    encrypt.init(Cipher.ENCRYPT_MODE,pubKey,sp);
    Cipher decrypt = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    decrypt.init(Cipher.DECRYPT_MODE,privkey,sp);

    byte[] A = encrypt.doFinal(content);
    byte[] B = decrypt.doFinal(A);

    String resultA = new String(A);
    String resultB = new String(B);
    Toast.makeText(getApplicationContext(), resultA + "\n" + resultB, Toast.LENGTH_LONG).show();
} catch (Exception e) {
    e.printStackTrace();
    Toast.makeText(getApplicationContext(), e.toString()+"\n"+e.getMessage(), Toast.LENGTH_LONG).show();
}

Answer 2

asymmetric key encryption available from android 6+( api 23+). This is a good guide for android's encryption. Additionsly after getBytes called you need to encode bytes with Base64.