OSX RSA Decryption from Bouncy Castle

Question

I'm attempting to decrypt data with a given private key from a Java process that is using BouncyCastle

The code that generates the private key is:

RSAPrivateCrtKeyParameters key = new RSAPrivateCrtKeyParameters(modulus, publicExponent, privateExponent, p, q, dP, dQ, qInv);
RSAPrivateKeyStructure struc = new RSAPrivateKeyStructure(key.getModulus(), key.getPublicExponent(), key.getExponent(), key.getP(), key.getQ(), key.getDP(), key.getDQ(), key.getQInv());
byte [] bytes = struc.getEncoded();

According to the documentation this should produce an RSA key in PKCS1 v2.1 format.

Here's an example output in base64

MIICXAIBAAKBgQC5THPoS5tXlIVMKjv2MwR1P3+pIOoWYT0M+uV/lmnDXRx02MrneKH0O0VZC1UgBZ4QKyAokl1oLwjfvBX3gmDzHdMJV8NWUAI09eRzqQEraXWsn1fjJBXbIU+OcIUl+tP0k59iLVMH6ztNnJH4JNWpxkUUXcN1VweOmyyRiTM+XQIDAQABAoGAITgQi3OKfM/2eaoUp9WLlg11MHfjH3H9ROdx30ay3UBDCruW3JpmxmcdzpHWh/AZ3yIsGLnsnvHXsGyCr6WjBYSTrWHBcAEgGgPIv8mW4Zhs0UjlrPMl6BzOJQRkgcnO549OjmEDymgM9Zjgc7PZIG9R4xzJC4MCHJElKNPuZy0CQQDfpB4jK8qZci8JWI7r403manN2tFDvlyQmx1wmuP4uLCSGlILYW/N1TYKesRLzsrDnSxFACev1nr5FN+Rtz3M/AkEA1BwYCbcJRXmTy5WvpzQ9GAzoSKt2TA1XKBDLzW+MQxX9+k0jzXSCsfMpJVv4HsAWTF9jK2elkw2dK10y9JQTYwJAH8C0irfX/ytoJEKPoExYriNlG3CEoU/rURozKkhPxox77yO85C1CBuCncBKiJzmyxlQxMc920hvk1f0WkMTIjwJBAI+myzzhIr1Tvp3VrZXfEjVUoIh+32pYapuu7Ggg0OyqW00Gq2W9L7qXBVnQ9LTWwQNOYtqFoqR6RuSUhSsRkiMCQA9q3UzFzXXcqmhhcXfzKHsMvHHr7WHYGz0YoqDi/4cmL50yA8oHBwY++rba8h6IdNAtT4VbQImbY6pS1Kq31ho=

This appears to be correct based on what I've read of PKCS1. I don't think the Java code is using padding -- at least I have not seen anything that says PKCS1, PKCS5, etc -- so I believe I must use "kSecPaddingNone" on iOS and Mac.

I'm attempting to decrypt this on both iOS and Mac OSX. I have it working on iOS with

status = SecKeyDecrypt(key, kSecPaddingNone, (uint8_t *)[data bytes], [data length], plainBuffer, &plainBufferSize);

The code actually works so I was happy... until I realized SecKeyDecrypt is not available on OSX. So I tried this...

// Create the SecKeyRef
CFMutableDictionaryRef parameters = CFDictionaryCreateMutable(kCFAllocatorDefault, 0, NULL, NULL);
CFDictionarySetValue(parameters, kSecAttrKeyType, kSecAttrKeyTypeRSA);
CFDictionarySetValue(parameters, kSecAttrKeyClass, kSecAttrKeyClassPrivate);
SecKeyRef key = SecKeyCreateFromData(parameters, (__bridge CFDataRef)keyData, &error);

// And decrypt.  Error handling removed from code segment
SecTransformRef decrypt = SecDecryptTransformCreate(key, &error);
SecTransformSetAttribute(decrypt, kSecPaddingKey, kSecPaddingNoneKey, &error);       // Sorta works
SecTransformSetAttribute(decrypt, kSecTransformInputAttributeName, (__bridge CFDataRef)encryptedData, &error);
CFDataRef decryptedData = SecTransformExecute(decrypt, &error);

This actually nearly works. If I input 128 bytes of encrypted data, it decrypts it correctly but pads the decrypted with 0 at the beginning of the block. So the actual data decrypted is 100 bytes. So I end up with

[ 28 bytes of 0s ][Correct decrypted data]

So a few questions. How can I tell how to many initial 0s to chop off from the decrypted data? Or is there some way to not pad the decrypted data?

Thanks

-- Update --

Okay was lurking in the Mac documentation and found a comment that said "No Padding" schema is not valid for encrypted data that begins with 0. So, I told our Java team we need to pad with:

    PKCS1Encoding eng = new PKCS1Encoding(new RSAEngine());
    eng.init(true, keys.getPublic());
    byte[] result = eng.processBlock(data, 0, data.length);

and I changed my iOS decryption code to

    status = SecKeyDecrypt( key, kSecPaddingPKCS1, ( uint8_t*)[ data bytes], [ data length], plainBuffer, &plainBufferSize);

and everything worked fine. So, I changed the OSX code to:

    SecTransformRef decrypt = SecDecryptTransformCreate(key, &error);
    SecTransformSetAttribute(decrypt, kSecPaddingKey, kSecPaddingPKCS1Key, &error);      
    SecTransformSetAttribute(decrypt, kSecTransformInputAttributeName, (__bridge CFDataRef)data, &error);
    NSData *decryptedData = CFBridgingRelease(SecTransformExecute(decrypt, &error));

and expected it to work... however it throws this error

Error Domain=NSOSStatusErrorDomain Code=-2147415748 "The operation couldn’t be completed. (OSStatus error -2147415748 - CSSMERR_CSP_INVALID_ATTR_PADDING)" UserInfo=0x6080002750c0 {NSDescription=CSSMERR_CSP_INVALID_ATTR_PADDING}

Any ideas?

-- Update 2 -- Here is a dumpasn1 of a sample private key

  0  604: SEQUENCE {
   4    1:   INTEGER 0
   7  129:   INTEGER
         :     00 85 ED 90 D9 01 6E 9A 55 97 54 8F 0F 18 CB B5
         :     E4 22 5F CE 8C 15 5E 1B 87 D3 95 4A 38 6D 22 69
         :     DC 66 47 21 08 0F 37 AD 7D 60 F9 56 8B 84 6F 73
         :     F6 37 63 45 61 E8 BF F7 D5 47 CE 5B 49 60 17 65
         :     6C FD 0F AF AD E9 65 AD D5 9C 7E 3F EF 7E 51 4A
         :     02 49 97 83 3A CF 24 B8 EF 50 8C 1D 00 B1 15 27
         :     6E BD 91 E5 62 B0 62 78 97 6F 08 5D 76 C9 86 74
         :     6C 7C 00 C7 CB 28 45 6C 96 F6 42 CD 83 05 F2 45
         :             [ Another 1 bytes skipped ]
 139    3:   INTEGER 65537
 144  128:   INTEGER
         :     18 95 79 27 3C 6A 0F 0E 73 0E E4 8B C2 E3 71 EA
         :     04 9D 4D 8E CD 45 4F 0C 69 BC 57 B9 6F DF 07 4B
         :     9B C2 A6 BF 91 FB 88 6F 21 63 E3 8D 0C AC 60 BE
         :     EB 7F DF 76 8F 80 DD 7F 5B 04 F8 20 C9 F0 C1 7F
         :     32 22 36 C1 F7 38 13 4B 40 F7 85 AC B2 3F 49 AC
         :     3A C7 43 89 BD 7B CE 7F BE 02 AD B6 04 8C 22 A0
         :     1F 64 98 36 D9 C4 BC 9C A7 9F 0C 86 48 29 57 13
         :     BD 36 97 BF 13 51 19 BE 8A C8 DD DD 0E 77 DE 11
 275   65:   INTEGER
         :     00 B9 51 A5 C1 29 A0 CF F6 3C 68 FF BC 94 60 A0
         :     43 65 56 8A BA 1E 11 7A A0 6D 76 66 7D C6 B2 34
         :     C9 1A 59 D4 13 96 F4 C5 E3 8A C8 E0 84 2D 78 9F
         :     46 0E 26 B9 DD 13 93 0A 12 34 DE 76 C9 B7 6D 67
         :     2F
 342   65:   INTEGER
         :     00 B9 02 28 A6 45 11 1F AF 5C 9E 7E 75 BC 30 2A
         :     29 06 FE 66 54 52 79 2C F4 02 92 09 92 FF 73 59
         :     E4 8A 8F B5 22 9F CC CF E9 78 52 4B EE B8 D5 33
         :     7B B6 B5 38 28 27 2A 0A AE 89 D2 21 65 C0 FC 88
         :     E5
 409   64:   INTEGER
         :     6B CC 2C A9 01 F8 03 40 6E BF 7D 13 4B 14 31 E5
         :     42 4B 67 03 00 7E 96 60 3F 8C 41 EE 23 E8 81 80
         :     01 8E 03 29 2A 04 54 20 1A 18 E3 50 BF CA 8C 8B
         :     89 AB C9 2D EA 36 FC 02 BF 32 30 D3 01 99 E8 0D
 475   64:   INTEGER
         :     79 08 D3 85 2B 6C 2F 79 6F 33 75 72 1A E2 BB C2
         :     49 84 07 78 24 D8 87 B3 3F 37 41 32 3D 12 BE FD
         :     88 34 CA 00 D3 E0 8F 28 A3 81 DB 91 5A B4 88 50
         :     E8 50 18 64 14 73 29 B7 D4 0C 77 B2 F5 15 81 8D
 541   65:   INTEGER
         :     00 A6 73 29 DC BC 09 91 FF 14 54 DA 80 20 94 80
         :     D6 D5 5D E0 84 2E C8 F4 F0 D5 27 90 9B C5 BE 4D
         :     48 C3 6A 2B 74 E7 16 E2 44 93 C3 33 FC AA DE CA
         :     5E 45 97 C3 B2 3F 7A FE 5A A6 F9 8F A2 7D B9 CF
         :     AD
         :   }

Answer 1

This was answered by Apple support. Search their CryptoCompatibility sample code for "rdar://problem/13661366" and you'll find this:

    // For an RSA key the transform does PKCS#1 padding by default.  Weirdly, if we explicitly
    // set the padding to kSecPaddingPKCS1Key then the transform fails <rdar://problem/13661366>>. 
    // Thus, if the client has requested PKCS#1, we leave paddingStr set to NULL, which prevents
    // us explicitly setting the padding to anything, which avoids the error while giving us
    // PKCS#1 padding.

So the fix is:

SecTransformSetAttribute(decrypt, kSecPaddingKey, NULL, &error);

OR

We actually ended up using OAEP padding