RSA Encryption Javascript and Decrypt Java

Question

Spent almost 2 days with different combinations.I am generating a asymmetric key pair (public and private) in java using RSA algorithm and trying to use the public key in javascript to encrypt some text and decrypt back in java on server side. I am getting "javax.crypto.IllegalBlockSizeException: Data must not be longer than 128 bytes" exception while trying to decrypt back the string encrypted in javascript. Would appreciate some help...

Using thi Javascript library to encrypt.

https://github.com/wwwtyro/cryptico

var publicKeyString = ""// base64encoded public key string generated in java

Here is my javascript code

var EncryptionResult = cryptico.encrypt("somestring", publicKeyString);
console.log("Encrypted status-"+EncryptionResult.status);
console.log("Encrypted String-"+EncryptionResult.cipher);

It is successfully encrypting the string.

Java Key Generation and Decryption

Cipher cipher = Cipher.getInstance("RSA");  
KeyFactory fact = KeyFactory.getInstance("RSA"); 
KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
keyPairGenerator.initialize(1024); // 1024 used for normal

KeyPair keyPair = keyPairGenerator.generateKeyPair();
PublicKey publicKey = keyPair.getPublic();
PrivateKey privateKey = keyPair.getPrivate();

FileOutputStream fos = null;
ObjectOutputStream oos = null;

Code to store the private key in file which is used to decrypt in decrypt method.

  RSAPrivateKeySpec rsaPrivKeySpec = fact.getKeySpec(privateKey, 
                                                     RSAPrivateKeySpec.class);
 System.out.println("Writing private key...");
 fos = new FileOutputStream(PRIVATE_KEY_FILE);
 oos = new ObjectOutputStream(new BufferedOutputStream(fos));
 oos = new ObjectOutputStream(new BufferedOutputStream(fos));
 oos.writeObject(rsaPrivKeySpec.getModulus());
 oos.writeObject(rsaPrivKeySpec.getPrivateExponent());
 oos.close();

Decrypt method

public String decrypt(String ciphertext)   
      throws IllegalBlockSizeException, BadPaddingException, InvalidKeyException   
       {  
             if (ciphertext.length() == 0) return null;  
             byte[] dec = org.apache.commons.codec.binary.Base64.decodeBase64(ciphertext);  
             try {
             System.out.println("Private Key file name----"+PRIVATE_KEY_FILE);
         privateKey = readPrivateKeyFromFile(PRIVATE_KEY_FILE);
        } catch (IOException e) {
        e.printStackTrace();
        }
             cipher.init(Cipher.DECRYPT_MODE, privateKey);  
             byte[] decrypted = cipher.doFinal(dec);  
             return new String(decrypted, PLAIN_TEXT_ENCODING);  
  } 

 //reading private key from file

public PrivateKey readPrivateKeyFromFile(String fileName)
    throws IOException {
  FileInputStream fis = null;
  ObjectInputStream ois = null;
   try {
    fis = new FileInputStream(new File(fileName));
    ois = new ObjectInputStream(fis);
    System.out.println("Private Key file-"+fileName);

    BigInteger modulus = (BigInteger) ois.readObject();
    BigInteger exponent = (BigInteger) ois.readObject();

    // Get Private Key
    RSAPrivateKeySpec rsaPrivateKeySpec = new RSAPrivateKeySpec(modulus, exponent);
    KeyFactory fact = KeyFactory.getInstance("RSA");
    PrivateKey privateKey = fact.generatePrivate(rsaPrivateKeySpec);
    return privateKey;
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        if (ois != null) {
            ois.close();
            if (fis != null) {
                fis.close();
            }
        }
    }
    return null;
 }    

Answer 1

From the Cryptico documentation it seems that it is not a simple RSA encryption, but a complex operation that generates AES key, encrypts it with RSA, encrypts the data with AES and outputs a concatenation of encrypted AES key and encrypted data. If you want to decrypt that in Java you will have to check the Cryptico source code and reimplement the same in Java.

As for your current attempt and javax.crypto.IllegalBlockSizeException: Data must not be longer than 128 bytes error:

When you do not specify the full transformation the default JCE transformation for RSA is RSA/ECB/PKCS1Padding.

In this mode the RSA encrypts or decrypts a single block of data which the length is not greater than the size of the key (more specifically, if the input sequence of bytes is interpreted as a big integer, its value should be less that the modulus used by the RSA). You can find additional information in this and this questions.

With the key size of 1024 bits the maximum data size is 128 bytes, and that is exactly what the exception says because the output of Cryptico is obviously not a single RSA block and its length is greater that expected by "plain" RSA. Trying to use some other cipher mode or padding mode in Java will not help in that situation either.

Answer 2

Thanks Oleg for the detailed information. I will definitely take a look into it.

For now I switched to jsencrypt and it seems to work fine.

https://github.com/travist/jsencrypt

EDIT

How you get the encoded public key for the js encrypt?