Private key signature different on iOS and MacOSX

Question

I implemented a category method on the NSData class which returns a signature of the data using an SHA-1 hash and subsequent encryption with a private key as follows:

- (NSData *)signatureWithKey:(SecKeyRef)keyRef {

    if (keyRef == NULL) {
        return nil;
    }

    NSData *sha1Digest = [self dataWithSHA1Digest];

    size_t maxLength = SecKeyGetBlockSize(keyRef) - 11;

    if ([sha1Digest length] > maxLength) {
        NSString *reason = [NSString stringWithFormat:@"Digest is too long to sign with this key, max length is %ld and actual length is %ld", maxLength, (unsigned long)[self length]];
        NSException *ex = [NSException exceptionWithName:@"BMInvalidArgumentException" reason:reason userInfo:nil];
        @throw ex;
    }

#if TARGET_OS_IPHONE
    OSStatus status = noErr;

    uint8_t *plainBuffer = (uint8_t *)[sha1Digest bytes];
    size_t plainBufferSize = [sha1Digest length];
    size_t cipherBufferSize = SecKeyGetBlockSize(keyRef);
    uint8_t *cipherBuffer = malloc(cipherBufferSize * sizeof(uint8_t));

    status = SecKeyRawSign(keyRef,
                           kSecPaddingPKCS1SHA1,
                           plainBuffer,
                           plainBufferSize,
                           &cipherBuffer[0],
                           &cipherBufferSize
                           );

    if (status == noErr) {
        return [NSData dataWithBytesNoCopy:cipherBuffer length:cipherBufferSize freeWhenDone:YES];
    }

    free(cipherBuffer);
    return nil;
#else
    CFErrorRef error = NULL;
    SecTransformRef signer = NULL;
    CFTypeRef signature = NULL;
    if ((signer = SecSignTransformCreate(keyRef, &error))) {
        if (SecTransformSetAttribute(
                                 signer,
                                 kSecTransformInputAttributeName,
                                 (CFDataRef)sha1Digest,
                                     &error)) {
            signature = SecTransformExecute(signer, &error);
        }
    }

    if (error) {
        LogWarn(@"Could not sign: %@", error);
        CFRelease(error);
    }

    if (signer) {
        CFRelease(signer);
    }

    if (signature) {
        NSData *data = [NSData dataWithData:(NSData *)signature];
        CFRelease(signature);
        return data;
    } else {
        return nil;
    }

#endif

}

Now the strange thing is that with the same private key (loaded from a p12 file) I get two different results for iOS and MacOSX when signing the same data. I am completely puzzled by this. You may notice the method above uses a different implementation for MacOSX using security transforms, but even if I use the iOS implementation on MacOSX (which gives a compile warning but works fine) I get the same result.

The method used for loading the private key from file is below:

+ (SecKeyRef)newPrivateKeyRefWithPassword:(NSString *)password fromData:(NSData *)data {
    NSMutableDictionary * options = [[NSMutableDictionary alloc] init];

    SecKeyRef privateKeyRef = NULL;

    // Set the public key query dictionary
    //change to your .pfx  password here
    [options setObject:password forKey:(id)kSecImportExportPassphrase];

    CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);

    OSStatus securityError = SecPKCS12Import((CFDataRef)data,
                                             (CFDictionaryRef)options, &items);

    if (securityError == noErr && CFArrayGetCount(items) > 0) {
        CFDictionaryRef identityDict = CFArrayGetValueAtIndex(items, 0);
        SecIdentityRef identityApp =
        (SecIdentityRef)CFDictionaryGetValue(identityDict,
                                             kSecImportItemIdentity);

        securityError = SecIdentityCopyPrivateKey(identityApp, &privateKeyRef);
        if (securityError != noErr) {
            privateKeyRef = NULL;
        }
    }
    [options release];
    if (items) CFRelease(items);
    return privateKeyRef;
}

And this is the test case I use. Notice that two different strings are printed on iOS and MacOSX:

    NSString *test = @"bla";
    NSData *testData = [test dataUsingEncoding:NSUTF8StringEncoding];

    NSString *p12Path= [[NSBundle mainBundle] pathForResource:@"private_key" ofType:@"p12"];

    NSData *p12Data = [NSData dataWithContentsOfFile:p12Path];

    SecKeyRef keyRef = [BMSecurityHelper newPrivateKeyRefWithPassword:@"xxxxxxxx" fromData:p12Data];

    NSData *signatureData = [testData signatureWithKey:keyRef];
    NSString *signatureString = [BMEncodingHelper base64EncodedStringForData:signatureData withLineLength:0];

    if (keyRef) CFRelease(keyRef);

    NSLog(@"signatureString: %@", signatureString);

Answer 1

It's always nice if you can answer your own question. I missed the following: under MacOSX the security transform also calculates the SHA-1 hash automatically, in contrast with the iOS implementation.

I fixed the problem by adding the following in the MacOSX implementation:

SecTransformSetAttribute(signer, kSecInputIsAttributeName, kSecInputIsDigest, &error)