AFNetworking problems with TLS Verification of a self signed server root CA

Question

This is a question that tries to both find solutions for my particular use case, and to document what I've tried to do for anyone else who is following this process.

We have a RESTful server and an iOS app. We have our own certificate authority and the server has a root certificate authority and a self signed certificate. We followed this process to generate the following files:

http://datacenteroverlords.com/2012/03/01/creating-your-own-ssl-certificate-authority/

rootCA.pem rootCA.key server.crt server.key

Only the server certificates are stored on our server, and as part of the SSL process the public keys are sent with the API calls for verification.

I've followed this process to use AFNetworking to use certificate pinning as well as public key pinning to verify our self signed certificates:

http://initwithfunk.com/blog/2014/03/12/afnetworking-ssl-pinning-with-self-signed-certificates/

We convert the .crt file to a .cer file (in DER format) according to this guide:

https://support.ssl.com/Knowledgebase/Article/View/19/0/der-vs-crt-vs-cer-vs-pem-certificates-and-how-to-convert-them

and include the .cer file (server.cer) in the iOS app bundle. This successfully allows our app to make GET/POST requests to our server. However, because our server certificate might expire or get reissued, we want to instead use the root CA, as done by people in this thread on AFNetworking:

https://github.com/AFNetworking/AFNetworking/issues/1944

Currently we've updated to AFNetworking 2.6.0 so our networking libraries should definitely include all the updates, include ones in this discussion:

https://github.com/AFNetworking/AFNetworking/issues/2744

The code used to create our security policy:

    var manager: AFHTTPRequestOperationManager = AFHTTPRequestOperationManager()
    manager.requestSerializer = AFJSONRequestSerializer() // force serializer to use JSON encoding

    let policy: AFSecurityPolicy = AFSecurityPolicy(pinningMode: AFSSLPinningMode.PublicKey)
    var data: [NSData] = [NSData]()
    for name: String in ["rootCA", "server"] {
        let path: String? = NSBundle.mainBundle().pathForResource(name, ofType: "cer")
        let keyData: NSData = NSData(contentsOfFile: path!)!
        data.append(keyData)
    }
    policy.pinnedCertificates = data
    policy.allowInvalidCertificates = true 
    policy.validatesDomainName = false 
    manager.securityPolicy = policy

With server.cer included, we're able to trust our server by pinning the public key (also tried AFSecurityPolicyPinningMode.Certificate); this worked because the exact certificate is included. However because we might change up the server.crt file that the server has, so we want to be able to do it with just rootCA.cer.

However, with just the rootCA included in the app bundle, this doesn't seem to work. Is it that the rootCA doesn't have enough information about the public key to verify the server certificate, which was signed with the root CA? The server.crt file may also have a changing CommonName.

Also, since my fluency in SSL terminology is pretty raw, if anyone has can clarify whether I'm asking the correct questions, that would be great. The specific questions are:

1. Am I generating the certificates correctly so that the server can prove its identity using the self signed server.crt file?
2. Is it possible to include only the rootCA.cer file into the bundle and be able to verify the leaf certificate server.crt? Will it be able to verify another server2.crt file signed by the same rootCA? Or should we include an intermediate cert between the rootCA and the leaf?
3. Is public key pinning or certificate pinning the right solution for this? Every forum and blog post I've read says yes, but even with the most updated AFNetworking library we haven't had any luck.
4. Does the server need to somehow send both the server.crt and the roomCA.pem signatures over?

Answer 1

With the help of a bunch of different SSL resources, I've found the solution to enabling the use of self signed certificates to validate a SSL enabled private server. I have also gotten a much much better understanding of SSL, existing iOS solutions, and the minor issues with each one that made it not work in my system. I'll attempt to outline all the resources that went into my solution and what small things made the difference.

We are still using AFNetworking and currently it is 2.6.0 which supposedly includes certificate pinning. This was the root of our problem; we were unable to verify the identity of our private server, which was sending down a leaf certificate signed by a self-signed CA root. In our iOS app, we bundle the self signed root certificate, which is then set as a trusted anchor by AFNetworking. However, because the server is a local server (hardware included with our product) the IP address is dynamic, so AFNetworking's certificate validation fails because we weren't able to disable the IP check.

To get to the root of the answer, we are using an AFHTTPSessionManager in order to implement a custom sessionDidReceiveAuthenticationChallengeCallback. (See: https://gist.github.com/r00m/e450b8b391a4bf312966) In that callback, we validate the server certificate using a SecPolicy that doesn't check for host name; see http://blog.roderickmann.org/2013/05/validating-a-self-signed-ssl-certificate-in-ios-and-os-x-against-a-changing-host-name/, which is an older implementation for NSURLConnection rather than NSURLSession.

The code:

Creating an AFHTTPSessionManager

    var manager: AFHTTPSessionManager = AFHTTPSessionManager()
    manager.requestSerializer = AFJSONRequestSerializer() // force serializer to use JSON encoding
    manager.setSessionDidReceiveAuthenticationChallengeBlock { (session, challenge, credential) -> NSURLSessionAuthChallengeDisposition in

        if self.shouldTrustProtectionSpace(challenge, credential: credential) {
            // shouldTrustProtectionSpace will evaluate the challenge using bundled certificates, and set a value into credential if it succeeds
            return NSURLSessionAuthChallengeDisposition.UseCredential
        }
        return NSURLSessionAuthChallengeDisposition.PerformDefaultHandling
    }

Implementation of custom validation

class func shouldTrustProtectionSpace(challenge: NSURLAuthenticationChallenge, var credential: AutoreleasingUnsafeMutablePointer<NSURLCredential?>) -> Bool {
    // note: credential is a reference; any created credential should be sent back using credential.memory

    let protectionSpace: NSURLProtectionSpace = challenge.protectionSpace
    var trust: SecTrustRef = protectionSpace.serverTrust!

    // load the root CA bundled with the app
    let certPath: String? = NSBundle.mainBundle().pathForResource("rootCA", ofType: "cer")
    if certPath == nil {
        println("Certificate does not exist!")
        return false
    }

    let certData: NSData = NSData(contentsOfFile: certPath!)!
    let cert: SecCertificateRef? = SecCertificateCreateWithData(kCFAllocatorDefault, certData).takeUnretainedValue()

    if cert == nil {
        println("Certificate data could not be loaded. DER format?")
        return false
    }

    // create a policy that ignores hostname
    let domain: CFString? = nil
    let policy:SecPolicy = SecPolicyCreateSSL(1, domain).takeRetainedValue() 

    // takes all certificates from existing trust
    let numCerts = SecTrustGetCertificateCount(trust)
    var certs: [SecCertificateRef] = [SecCertificateRef]()
    for var i = 0; i < numCerts; i++ {
        let c: SecCertificateRef? = SecTrustGetCertificateAtIndex(trust, i).takeUnretainedValue()
        certs.append(c!)
    }

    // and adds them to the new policy
    var newTrust: Unmanaged<SecTrust>? = nil
    var err: OSStatus = SecTrustCreateWithCertificates(certs, policy, &newTrust)
    if err != noErr {
        println("Could not create trust")
    }
    trust = newTrust!.takeUnretainedValue() // replace old trust

    // set root cert
    let rootCerts: [AnyObject] = [cert!]
    err = SecTrustSetAnchorCertificates(trust, rootCerts)

    // evaluate the certificate and product a trustResult
    var trustResult: SecTrustResultType = SecTrustResultType()
    SecTrustEvaluate(trust, &trustResult)

    if Int(trustResult) == Int(kSecTrustResultProceed) || Int(trustResult) == Int(kSecTrustResultUnspecified) {
        // create the credential to be used
        credential.memory = NSURLCredential(trust: trust)
        return true
    }
    return false
}

A few things I learned about swift while going through this code.

1. AFNetworking's implementation of setSessionDidReceiveAuthenticationChallengeBlock has this signature:

   • (void)setSessionDidReceiveAuthenticationChallengeBlock:(nullable NSURLSessionAuthChallengeDisposition (^)(NSURLSession *session, NSURLAuthenticationChallenge *challenge, NSURLCredential * __nullable __autoreleasing * __nullable credential))block;

The credential parameter is a reference/inout variable that needs to be assigned. In swift it looks like this: AutoreleasingUnsafeMutablePointer. In order to assign something to it in C, you'd do something like this:

*credential = [[NSURLCredential alloc] initWithTrust...];

In swift, it looks like this: (from converting NSArray to RLMArray with RKValueTransFormer fails converting outputValue to AutoreleasingUnsafeMutablePointer<AnyObject?>)

credential.memory = NSURLCredential(trust: trust)

2. SecPolicyCreateSSL, SecCertificateCreateWithData and SecTrustGetCertificateAtIndex return Unmanaged! objects, you have to essentially convert them/bridge them using takeRetainedValue() or takeUnretainedValue(). (See http://nshipster.com/unmanaged/). We had memory issues/crashes when we used takeRetainedValue() and called the method more than once (there was crash on SecDestroy). Right now the build seems stable after we switched to using takeUnretainedValue(), since you don't need the certificates or ssl policies after the validation.

3. TLS sessions cache. https://developer.apple.com/library/ios/qa/qa1727/_index.html That means when you get a successful verification on a challenge, you never get the challenge again. This can really mess with your head when you're testing a valid certificate, then test an invalid certificate, which then skips all validation, and you get a successful response from the server. The solution is to Product->Clean in your iOS simulator after each time you use a valid certificate and pass the validation challenge. Otherwise you might spend some time thinking incorrectly that you finally got the root CA to validate.

So here's simply a working solution for the issues I was having with my servers. I wanted to post everything on here to hopefully help someone else who's running a local or dev server with a self signed CA and an iOS product that needs to be SSL enabled. Of course, with ATS in iOS 9 I expect to be digging into SSL very soon again.

This code currently has some memory management issues and will be updated in the near future. Also, if anyone sees this implementation and says "Ah hah, this is just as bad as returning TRUE for invalid certificates", please let me know! As far as I can tell through our own testing, the app rejects invalid server certificates not signed by our root CA, and accepts the leaf certificate generated and signed by the root CA. The app bundle only has the root CA included, so the server certificate can be cycled after they expire and existing apps won't fail.

If I dig into AFNetworking a little bit more and figure out a one-to-three line solution to all of this (by toggling all those little flags they provide) I'll also post an update.

If AlamoFire starts supporting SSL also feel free to post a solution here.

Answer 2

If you are using coco pods then subclass the AFSecurityPolicy class and implement the security check according to mitrenegade's answer https://stackoverflow.com/a/32469609/4000434

Hear is my code.

Initialise the AFHttpRequestOperationManager while posting request like below.

AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];
    manager.responseSerializer = [AFJSONResponseSerializer serializer];
    manager.requestSerializer = [AFJSONRequestSerializer serializer];
    manager.securityPolicy = [RootCAAFSecurityPolicy policyWithPinningMode:AFSSLPinningModeCertificate];
    [manager.requestSerializer setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    [manager POST:Domain_Name parameters:parameters success:^(AFHTTPRequestOperation *operation, id responseObject) {
        success(operation,responseObject);
        [[UIApplication sharedApplication] setNetworkActivityIndicatorVisible:NO];
    } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
        [[UIApplication sharedApplication] setNetworkActivityIndicatorVisible:NO];
        NSLog(@"Error  %@",error);
        failure(operation,error);
    }];

RootCAAFSecurityPolicy is the subclass of AFSecurityPolicy Class. See below for RootCAAFSecurityPolicy .h and .m class override the method

-(BOOL)evaluateServerTrust:(SecTrustRef)serverTrust forDomain:(NSString *)domain

RootCAAFSecurityPolicy.h class

#import <AFNetworking/AFNetworking.h>

@interface RootCAAFSecurityPolicy : AFSecurityPolicy

@end

RootCAAFSecurityPolicy.m class

Replace RootCA with your certificate file name

#import "RootCAAFSecurityPolicy.h"

@implementation RootCAAFSecurityPolicy
-(BOOL)evaluateServerTrust:(SecTrustRef)serverTrust forDomain:(NSString *)domain
{
    if(self.SSLPinningMode == AFSSLPinningModeCertificate)
    {
        return [self shouldTrustServerTrust:serverTrust];
    }
    else
    {
        return [super evaluateServerTrust:serverTrust forDomain:domain];
    }
}
- (BOOL)shouldTrustServerTrust:(SecTrustRef)serverTrust
{
    // load up the bundled root CA
    NSString *certPath = [[NSBundle mainBundle] pathForResource:@"RootCA" ofType:@"der"];

    NSAssert(certPath != nil, @"Specified certificate does not exist!");

    NSData *certData = [[NSData alloc] initWithContentsOfFile:certPath];
    CFDataRef certDataRef = (__bridge_retained CFDataRef)certData;
    SecCertificateRef cert = SecCertificateCreateWithData(NULL, certDataRef);

    NSAssert(cert != NULL, @"Failed to create certificate object. Is the certificate in DER format?");


    // establish a chain of trust anchored on our bundled certificate
    CFArrayRef certArrayRef = CFArrayCreate(NULL, (void *)&cert, 1, NULL);
    OSStatus anchorCertificateStatus = SecTrustSetAnchorCertificates(serverTrust, certArrayRef);

    NSAssert(anchorCertificateStatus == errSecSuccess, @"Failed to specify custom anchor certificate");


    // trust also built-in certificates besides the specified CA
    OSStatus trustBuiltinCertificatesStatus = SecTrustSetAnchorCertificatesOnly(serverTrust, false);

    NSAssert(trustBuiltinCertificatesStatus == errSecSuccess, @"Failed to reenable trusting built-in anchor certificates");


    // verify that trust
    SecTrustResultType trustResult;
    OSStatus evalStatus =  SecTrustEvaluate(serverTrust, &trustResult);

    NSAssert(evalStatus == errSecSuccess, @"Failed to evaluate certificate trust");


    // clean up
    CFRelease(certArrayRef);
    CFRelease(cert);
    CFRelease(certDataRef);


    // did our custom trust chain evaluate successfully
    return (trustResult == kSecTrustResultProceed || trustResult == kSecTrustResultUnspecified);
}
@end

Answer 3

I had the same problem and I've fixed it by comparing the public keys of the chain in the didReceiveChallenge method of the AFURLSessionManager.

-(void)URLSession:(NSURLSession *)session didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential * _Nullable))completionHandler {
        // Get remote certificate
        SecTrustRef serverTrust = challenge.protectionSpace.serverTrust;

        NSMutableArray *policies = [NSMutableArray array];
        [policies addObject:(__bridge_transfer id)SecPolicyCreateSSL(true, (__bridge CFStringRef) challenge.protectionSpace.host)];

        SecTrustSetPolicies(serverTrust, (__bridge CFArrayRef)policies);
        NSUInteger trustedPublicKeyCount = 0;
        NSArray *publicKeys = AFPublicKeyTrustChainForServerTrust(serverTrust);

        for (id trustChainPublicKey in publicKeys) {
            for (id pinnedPublicKey in self.pinnedPublicKeys) {
                if (AFSecKeyIsEqualToKey((__bridge SecKeyRef)trustChainPublicKey, (__bridge SecKeyRef)pinnedPublicKey)) {
                    trustedPublicKeyCount += 1;
                }
            }
        }

        // The pinnning check
        if (trustedPublicKeyCount > 0) {
            NSURLCredential *credential = [NSURLCredential credentialForTrust:serverTrust];
            completionHandler(NSURLSessionAuthChallengeUseCredential, credential);
        } else {
            completionHandler(NSURLSessionAuthChallengeCancelAuthenticationChallenge, NULL);
        }
    }

Here is the initialisation of pinnedPublicKeys:

    // Get local certificates
    NSArray *certNames = @[@"root_cert"];
    self.pinnedPublicKeys = [NSMutableSet new];

    for (NSString *certName in certNames) {
        NSString *path = [bundle pathForResource:certName ofType:@"der"];
        NSData *certificate = [NSData dataWithContentsOfFile:path];

        id publicKey = AFPublicKeyForCertificate(certificate);
        if (publicKey) {
            [self.pinnedPublicKeys addObject:publicKey];
        }
    }

Here are the helper Methods to get key trust chain (AFPublicKeyTrustChainForServerTrust), comparing the public keys (AFSecKeyIsEqualToKey) and the Method to get the public key from a certificate (AFPublicKeyTrustChainForServerTrust):

static NSArray * AFPublicKeyTrustChainForServerTrust(SecTrustRef serverTrust) {
    SecPolicyRef policy = SecPolicyCreateBasicX509();
    CFIndex certificateCount = SecTrustGetCertificateCount(serverTrust);
    NSMutableArray *trustChain = [NSMutableArray arrayWithCapacity:(NSUInteger)certificateCount];
    for (CFIndex i = 0; i < certificateCount; i++) {
        SecCertificateRef certificate = SecTrustGetCertificateAtIndex(serverTrust, i);

        SecCertificateRef someCertificates[] = {certificate};
        CFArrayRef certificates = CFArrayCreate(NULL, (const void **)someCertificates, 1, NULL);

        SecTrustRef trust;
        SecTrustCreateWithCertificates(certificates, policy, &trust);

        SecTrustResultType result;
        SecTrustEvaluate(trust, &result);

        [trustChain addObject:(__bridge_transfer id)SecTrustCopyPublicKey(trust)];

        if (trust) {
            CFRelease(trust);
        }

        if (certificates) {
            CFRelease(certificates);
        }

        continue;
    }
    CFRelease(policy);

    return [NSArray arrayWithArray:trustChain];
}

static BOOL AFSecKeyIsEqualToKey(SecKeyRef key1, SecKeyRef key2) {
    return [(__bridge id)key1 isEqual:(__bridge id)key2];
}

static id AFPublicKeyForCertificate(NSData *certificate) {
    id allowedPublicKey = nil;
    SecCertificateRef allowedCertificate;
    SecCertificateRef allowedCertificates[1];
    CFArrayRef tempCertificates = nil;
    SecPolicyRef policy = nil;
    SecTrustRef allowedTrust = nil;
    SecTrustResultType result;

    allowedCertificate = SecCertificateCreateWithData(NULL, (__bridge CFDataRef)certificate);

    allowedCertificates[0] = allowedCertificate;
    tempCertificates = CFArrayCreate(NULL, (const void **)allowedCertificates, 1, NULL);

    policy = SecPolicyCreateBasicX509();
    SecTrustCreateWithCertificates(tempCertificates, policy, &allowedTrust);
    SecTrustEvaluate(allowedTrust, &result);

    allowedPublicKey = (__bridge_transfer id)SecTrustCopyPublicKey(allowedTrust);

    if (allowedTrust) {
        CFRelease(allowedTrust);
    }

    if (policy) {
        CFRelease(policy);
    }

    if (tempCertificates) {
        CFRelease(tempCertificates);
    }

    if (allowedCertificate) {
        CFRelease(allowedCertificate);
    }

    return allowedPublicKey;
}