RSA: encrypt in iOS, decrypt in Java

Question

I have a public key that's sent from a Java server. The base64 encoded strings match before I decode and strip out the ASN.1 headers. I store the public key in the keychain with SecItemAdd.

So I'm trying to encrypt the data using the public key and decrypt it with the private key in Java. I'm using SecKeyEncrypt on the iOS side and Cipher on the Java side.

What I'm encrypting is the symmetric AES key that encrypts my actual data, so the key length is 16 bytes. When simply base64 encoding the key, everything works, so I know something is wrong with this RSA encryption.

Here's an example of my iOS call:

OSStatus sanityCheck = SecKeyEncrypt(publicKey,
        kSecPaddingPKCS1,
        (const uint8_t *) [incomingData bytes],
        keyBufferSize,
        cipherBuffer,
        &cipherBufferSize
);

Here's an example of my Java call:

public static byte[] decryptMessage (byte[] message, PrivateKey privateKey, String algorithm) {
    if (message == null || privateKey == null) {
        return null;
    }
    Cipher cipher = createCipher(Cipher.DECRYPT_MODE, privateKey, algorithm, false);
    if (cipher == null) {
        return null;
    }

    try {
        return cipher.doFinal(message);
    }
    catch (IllegalBlockSizeException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
    catch (BadPaddingException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
}

private static Cipher createCipher (int mode, Key encryptionKey, String algorithm, boolean useBouncyCastle) {
    Cipher cipher;

    try {
        if (useBouncyCastle) {
            Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
            cipher = Cipher.getInstance(algorithm, "BC");
        }
        else {
            cipher = Cipher.getInstance(algorithm);
        }
    }
    catch (NoSuchAlgorithmException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
    catch (NoSuchPaddingException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
    catch (NoSuchProviderException e) {
        e.printStackTrace();
        return null;
    }

    try {
        cipher.init(mode, encryptionKey);
    }
    catch (InvalidKeyException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }

    return cipher;
}

I've tried so many combinations and nothing has worked.

• iOS: PKCS1, Java: RSA/ECB/PKCS1Padding
• iOS: PKCS1, Java: RSA
• iOS: PKCS1, Java: RSA/None/PKCS1Padding (throws org.bouncycastle.crypto.DataLengthException: input too large for RSA cipher.)
• iOS: OAEP, Java: RSA/ECB/OAEPWithSHA-1AndMGF1Padding
• iOS: OAEP, Java: RSA/ECB/OAEPWithSHA-256AndMGF1Padding

I've also tried using the internal Java provider as well as the BouncyCastle provider. The javax.crypto.BadPaddingException gets thrown every time, but the message is different for each combination. Some show Data must start with zero, while others are Message is larger than modulus.

The iOS: PKCS1, Java: RSA doesn't throw an exception, but the resulting decrypted byte[] array should be length 16, but it's length 256, which means the padding isn't correctly stripped out.

Can someone help?

*** EDIT ***

As I'm doing more testing, I came across this page (http://javadoc.iaik.tugraz.at/iaik_jce/current/iaik/pkcs/pkcs1/RSACipher.html), which essentially tells me that RSA == RSA/None/PKCS1Padding. The decryption works in the sense that there are no exceptions, but I'm still getting a decrypted key whose byte[] is length 256 instead of length 16.

Another point of interest. It seems that if the Java server has the public key generated from the iOS device and encrypted using Cipher.getInstance("RSA"), the phone is able to decode the message correctly with RSA/PKCS1.

*** EDIT 2 ***

I have looked at these tutorials and looked through my code again on the iOS side:

• http://blog.flirble.org/2011/01/05/rsa-public-key-openssl-ios/
• http://blog.wingsofhermes.org/?p=42
• http://blog.wingsofhermes.org/?p=75

As far as I can tell, my code is doing everything correctly. One significant difference was in how I was saving the key, so I tried saving it the other way:

    OSStatus error = noErr;
    CFTypeRef persistPeer = NULL;

    NSMutableDictionary * keyAttr = [[NSMutableDictionary alloc] init];

    keyAttr[(__bridge id) kSecClass] = (__bridge id) kSecClassKey;
    keyAttr[(__bridge id) kSecAttrKeyType] = (__bridge id) kSecAttrKeyTypeRSA;
    keyAttr[(__bridge id) kSecAttrApplicationTag] = [secKeyWrapper getKeyTag:serverPublicKeyTag];
    keyAttr[(__bridge id) kSecValueData] = strippedServerPublicKey;
    keyAttr[(__bridge id) kSecReturnPersistentRef] = @YES;

    error = SecItemAdd((__bridge CFDictionaryRef) keyAttr, (CFTypeRef *)&persistPeer);

    if (persistPeer == nil || ( error != noErr && error != errSecDuplicateItem)) {
        NSLog(@"Problem adding public key to keychain");
        return;
    }

    CFRelease(persistPeer);

That save was successful, but the end result was the same: the decrypted AES key was still 256 bytes long instead of 16 bytes.

Answer 1

I had same problem. Does work with kSecPaddingNone, but doesn't work with kSecPaddingPKCS1 with any PKCS1 combination in Java code.

But, it's not good idea to use it without padding.

So, on iOS, replace kSecPaddingNone with kSecPaddingOAEP and use RSA/NONE/OAEPWithSHA1AndMGF1Padding in your Java code. This does work for me.

Answer 2

Solution with RSA/None/NoPadding

Okay, so I got it working but WITHOUT PADDING. This part is really frustrating me and I leave it up to others to try to help out. Maybe I'll eventually release what I have as a library on github, one for Obj-C, one for Java. Here's what I found out so far.

TL;DR: save the key to the keychain with minimal attributes to make retrieval simpler. Encrypt with SecKeyEncrypt but use kSecPaddingNone. Decrypt on Java side with BouncyCastle and algorithm RSA/None/NoPadding.

Sending RSA Public Key to iOS from Java

Using X.509 Certificate

I wanted to verify whether sending the public key directly, stripping out the ASN.1 header and saving was actually doing what it was supposed to do. So I looked at sending the public key over as a certificate instead. I want to give credit to David Benko for providing an encryption library (https://github.com/DavidBenko/DBTransitEncryption) that helped me with the certificate conversion. I didn't actually use his library because 1. I'm already using RNCryptor/JNCryptor for my AES encryption and 2. he doesn't have a Java side component, so I would need to write my own AES decryption there and I didn't want to do that. For those interested and want to take this approach, here's my code for creating a certificate on the Java side and then converting that certificate to a public key on iOS:

* Important Note: Please replace e.printStackTrace() with real logging statements. I only used this for testing and NOT in production.

Java:

public static X509Certificate generateCertificate (KeyPair newKeys) {
    Security.addProvider(new BouncyCastleProvider());
    Date startDate = new Date();
    Date expiryDate = new DateTime().plusYears(100).toDate();

    BigInteger serialNumber = new BigInteger(10, new Random());
    try {
        ContentSigner sigGen = new JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(newKeys
                                                                                                          .getPrivate());
        SubjectPublicKeyInfo subjectPublicKeyInfo = new SubjectPublicKeyInfo(ASN1Sequence.getInstance(newKeys
                                                                                                              .getPublic().getEncoded()
                                                                                                              ));
        X500Name dnName = new X500Name("CN=FoodJudge API Certificate");
        X509v1CertificateBuilder builder = new X509v1CertificateBuilder(dnName,
                                                                        serialNumber,
                                                                        startDate, expiryDate,
                                                                        dnName,
                                                                        subjectPublicKeyInfo);
        X509CertificateHolder holder = builder.build(sigGen);
        return new JcaX509CertificateConverter().setProvider("BC").getCertificate(holder);
    }
    catch (OperatorCreationException e) {
        e.printStackTrace();
    }
    catch (CertificateException e) {
        e.printStackTrace();
    }
    return null;
}

Obj-C:

- (SecKeyRef)extractPublicKeyFromCertificate:(NSData *)certificateBytes {
    if (certificateBytes == nil) {
        return nil;
    }

    SecCertificateRef certificate = SecCertificateCreateWithData(kCFAllocatorDefault, ( __bridge CFDataRef) certificateBytes);
    if (certificate == nil) {
        NSLog(@"Can not read certificate from data");
        return false;
    }

    SecTrustRef trust;
    SecPolicyRef policy = SecPolicyCreateBasicX509();
    OSStatus returnCode = SecTrustCreateWithCertificates(certificate, policy, &trust);

    // release the certificate as we're done using it
    CFRelease(certificate);
    // release the policy
    CFRelease(policy);

    if (returnCode != errSecSuccess) {
        NSLog(@"SecTrustCreateWithCertificates fail. Error Code: %d", (int)returnCode);
        return nil;
    }

    SecTrustResultType trustResultType;
    returnCode = SecTrustEvaluate(trust, &trustResultType);
    if (returnCode != errSecSuccess) {
        // TODO log
        CFRelease(trust);
        return nil;
    }

    SecKeyRef publicKey = SecTrustCopyPublicKey(trust);
    CFRelease(trust);

    if (publicKey == nil) {
        NSLog(@"SecTrustCopyPublicKey fail");
        return nil;
    }

    return publicKey;
}

Using RSA Public Key

It's important to note that you don't need send the public key over as a certificate. In fact, after discovering that the public key was being saved incorrectly (see below), I reverted this code and saved the public key to my device. You'll need to strip the ASN.1 header as mentioned in one of the blog posts. That code is reposted here (formatted for clarity).

+ (NSData *)stripPublicKeyHeader:(NSData *)keyBits {
    // Skip ASN.1 public key header
    if (keyBits == nil) {
        return nil;
    }

    unsigned int len = [keyBits length];
    if (!len) {
        return nil;
    }

    unsigned char *c_key = (unsigned char *)[keyBits bytes];
    unsigned int  idx    = 0;

    if (c_key[idx++] != 0x30) {
        return nil;
    }

    if (c_key[idx] > 0x80) {
        idx += c_key[idx] - 0x80 + 1;
    }
    else {
        idx++;
    }

    if (idx >= len) {
        return nil;
    }

    if (c_key[idx] != 0x30) {
        return nil;
    }

    idx += 15;

    if (idx >= len - 2) {
        return nil;
    }

    if (c_key[idx++] != 0x03) {
        return nil;
    }

    if (c_key[idx] > 0x80) {
        idx += c_key[idx] - 0x80 + 1;
    }
    else {
        idx++;
    }

    if (idx >= len) {
        return nil;
    }

    if (c_key[idx++] != 0x00) {
        return nil;
    }

    if (idx >= len) {
        return nil;
    }

    // Now make a new NSData from this buffer
    return([NSData dataWithBytes:&c_key[idx] length:len - idx]);
}

So I would simply save the key like so:

- (void)storeServerPublicKey:(NSString *)serverPublicKey {
    if (!serverPublicKey) {
        return;
    }
    SecKeyWrapper *secKeyWrapper = [SecKeyWrapper sharedWrapper];
    NSData *decryptedServerPublicKey = [[NSData alloc] initWithBase64EncodedString:serverPublicKey options:0];

    NSData *strippedServerPublicKey = [SecKeyWrapper stripPublicKeyHeader:decryptedServerPublicKey];
    if (!strippedServerPublicKey) {
        return;
    }
    [secKeyWrapper savePublicKeyToKeychain:strippedServerPublicKey tag:@"com.sampleapp.publickey"];
}

Saving RSA Public Key to Keychain

It was maddening. It turned out that even though I saved my key to the keychain, what I retrieved wasn't what I put in! I found this out by accident when I was comparing the base64 key that I was saving to the base64 key that I was using to encrypt my AES key. So I found out that it's better to simplify the NSDictionary used when saving the key. Here's what I ended up with:

- (void)savePublicKeyToKeychain:(NSData *)key tag:(NSString *)tagString {
    NSData *tag = [self getKeyTag:tagString];

    NSDictionary *saveDict = @{
            (__bridge id) kSecClass : (__bridge id) kSecClassKey,
            (__bridge id) kSecAttrKeyType : (__bridge id) kSecAttrKeyTypeRSA,
            (__bridge id) kSecAttrApplicationTag : tag,
            (__bridge id) kSecAttrKeyClass : (__bridge id) kSecAttrKeyClassPublic,
            (__bridge id) kSecValueData : key
    };
    [self saveKeyToKeychain:saveDict tag:tagString];
}

- (void)saveKeyToKeychain:(NSDictionary *)saveDict tag:(NSString *)tagString {
    OSStatus sanityCheck = SecItemAdd((__bridge CFDictionaryRef) saveDict, NULL);
    if (sanityCheck != errSecSuccess) {
        if (sanityCheck == errSecDuplicateItem) {
            // delete the duplicate and save again
            sanityCheck = SecItemDelete((__bridge CFDictionaryRef) saveDict);
            sanityCheck = SecItemAdd((__bridge CFDictionaryRef) saveDict, NULL);
        }
        if (sanityCheck != errSecSuccess) {
            NSLog(@"Problem saving the key to keychain, OSStatus == %d.", (int) sanityCheck);
        }
    }
    // remove from cache
    [keyCache removeObjectForKey:tagString];
}

To retrieve my key, I use the following methods:

 - (SecKeyRef)getKeyRef:(NSString *)tagString isPrivate:(BOOL)isPrivate {
     NSData *tag = [self getKeyTag:tagString];

     id keyClass = (__bridge id) kSecAttrKeyClassPublic;
     if (isPrivate) {
         keyClass = (__bridge id) kSecAttrKeyClassPrivate;
     }

     NSDictionary *queryDict = @{
             (__bridge id) kSecClass : (__bridge id) kSecClassKey,
             (__bridge id) kSecAttrKeyType : (__bridge id) kSecAttrKeyTypeRSA,
             (__bridge id) kSecAttrApplicationTag : tag,
             (__bridge id) kSecAttrKeyClass : keyClass,
             (__bridge id) kSecReturnRef : (__bridge id) kCFBooleanTrue
     };
     return [self getKeyRef:queryDict tag:tagString];
 }

- (SecKeyRef)getKeyRef:(NSDictionary *)query tag:(NSString *)tagString {
    SecKeyRef keyReference = NULL;
    OSStatus sanityCheck = SecItemCopyMatching((__bridge CFDictionaryRef) query, (CFTypeRef *) &keyReference);
    if (sanityCheck != errSecSuccess) {
        NSLog(@"Error trying to retrieve key from keychain. tag: %@. sanityCheck: %li", tagString, sanityCheck);
        return nil;
    }
    return keyReference;
}

At the end of the day, I was able to only get it working without padding. I'm not sure why BouncyCastle couldn't remove the padding, so if anyone has any insight, let me know.

Here's my code for encrypting (modified from David Benko):

- (NSData *)encryptData:(NSData *)content usingPublicKey:(NSString *)publicKeyTag {
    SecKeyRef publicKey = [self getKeyRef:publicKeyTag isPrivate:NO];
    NSData *keyBits = [self getKeyBitsFromKey:publicKey];
    NSString *keyString = [keyBits base64EncodedStringWithOptions:0];
    NSAssert(publicKey != nil,@"Public key can not be nil");

    size_t cipherLen = SecKeyGetBlockSize(publicKey); // convert to byte
    void *cipher = malloc(cipherLen);
    size_t maxPlainLen = cipherLen - 12;

    size_t plainLen = [content length];
    if (plainLen > maxPlainLen) {
        NSLog(@"content(%ld) is too long, must < %ld", plainLen, maxPlainLen);
        return nil;
    }

    void *plain = malloc(plainLen);
    [content getBytes:plain
               length:plainLen];

    OSStatus returnCode = SecKeyEncrypt(publicKey, kSecPaddingNone, plain,
            plainLen, cipher, &cipherLen);

    NSData *result = nil;
    if (returnCode != errSecSuccess) {
        NSLog(@"SecKeyEncrypt fail. Error Code: %d", (int)returnCode);
    }
    else {
        result = [NSData dataWithBytes:cipher
                                length:cipherLen];
    }

    free(plain);
    free(cipher);

    return result;
}

Here's how I decrypt on the Java side:

private Response authenticate (String encryptedSymmetricString) {
    byte[] encryptedSymmetricKey = Base64.decodeBase64(encryptedSymmetricKeyString);
    String privateKey = Server.getServerPrivateKey();
    byte[] decryptedSymmetricKey = KeyHandler.decryptMessage(encryptedSymmetricKey, privateKey,
                                                             KeyHandler.ASYMMETRIC_CIPHER_ALGORITHM);
}

public static byte[] decryptMessage (byte[] message, String privateKeyString, String algorithm) {
    if (message == null || privateKeyString == null) {
        return null;
    }
    PrivateKey privateKey = getPrivateKey(privateKeyString);
    return decryptMessage(message, privateKey, algorithm);
}

public static byte[] decryptMessage (byte[] message, PrivateKey privateKey, String algorithm) {
    if (message == null || privateKey == null) {
        return null;
    }
    Cipher cipher = createCipher(Cipher.DECRYPT_MODE, privateKey, algorithm, true);
    if (cipher == null) {
        return null;
    }

    try {
        return cipher.doFinal(message);
    }
    catch (IllegalBlockSizeException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
    catch (BadPaddingException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        return null;
    }
}