NSURLCache, together with NSURLSession, does not respect: Cache-Control: max-age:86000, private, must-revalidate

Question

In AppDelegate.m, I configured:

NSURLCache *sharedURLCache = [[NSURLCache alloc] initWithMemoryCapacity:20 * 1024 * 1024 diskCapacity:100 * 1024 * 1024 diskPath:@"FhtHttpCacheDir"];

Then the http request:

- (void) testRestfulAPI{
    NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
    NSURLSession *session = [NSURLSession sessionWithConfiguration:config];

    NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:@"http://192.168.0.223:8000/v1/topictypes"]];

    [request setHTTPMethod:@"GET"];
    [request setValue:@"application/json" forHTTPHeaderField:@"Accept"];

    NSError *error = nil;
    if (!error) {
        NSURLSessionDataTask *downloadTask = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
            if (!error) {
                NSHTTPURLResponse *httpResp = (NSHTTPURLResponse*) response;
                if (httpResp.statusCode == 200) {
                    NSDictionary* json = [NSJSONSerialization
                                          JSONObjectWithData:data
                                          options:kNilOptions
                                          error:&error];
                    NSLog(@"JSON: %@", json);
                }
            }
        }];
        [downloadTask resume];
    }
}

The first time it requests, it got HTTP 200 with Etag + Cache-Control headers. No problem.

If I am not wrong, Cache-Control: must-revalidate, max-age=86400, private will tell NSURLCache to consider the cache as being fresh within 24 hours and will not make any network calls within the next 24 hours.

But it is not the case, the second time the http request is made, it actually sends out If-None-Match headers out and got back HTTP 304.

It appears to me that NSURLCache is partially working. It can cache response, but it does not respect RFC 2616 semantics as Apple doc describes so here. FYI, I did not change the cache policy so it uses the default NSURLRequestUseProtocolCachePolicy.

I googled for more than a day for similar issues and other experienced similar ones but I have not found any solutions. Some asked about the same problem in AFNetworking's github issues but the author closes the issue as it is not directly related to AFNetworking here and here.

Also various related stackoverflow posts did not help me either.

Answer 1

Problem

The problem is the usage of the Cache-Control response directive must-revalidate.

By omitting must-revalidate you already have the perfect definition of your use case as far as I've understood it:

Cache-Control: max-age=86400, private

This controls how long the requested resource is considered fresh. After this time has elapsed, the answer should no longer come directly from the cache instead the server should be contacted for validation for subsequent requests. In your case since the server supplies an ETag, iOS sends a request with an If-None-Match header to the server.

Verification

To check this, I used your testRestfulAPI method without NSURLCache settings and configured a maximum age of 60 seconds on the server side, so I don't have to wait a day to check the result.

After that, I triggered testRestfulAPI once per second. I always got the desired result from the cache. And Charles showed that the data must come from the cache because the server was not contacted for 60 seconds.

RFC 7234

Here is a quote from RFC 7234 (which obsoletes RFC 2616), under 5.2.2.1. it states:

The must-revalidate directive is necessary to support reliable operation for certain protocol features. In all circumstances a cache MUST obey the must-revalidate directive; in particular, if a cache cannot reach the origin server for any reason, it MUST generate a 504 (Gateway Timeout) response.

The must-revalidate directive ought to be used by servers if and only if failure to validate a request on the representation could result in incorrect operation, such as a silently unexecuted financial transaction.

After reading that and if you put yourself in the view of a cache developer, you can well imagine that when a must-revalidate is seen, the original server is always contacted and any additional directives such as max-age are simply ignored. It seems to me that caches often show exactly this behavior in practice.

There is another section in chapter 5.2.2.1. which I will not conceal and which reads as follows:

The "must-revalidate" response directive indicates that once it has become stale, a cache MUST NOT use the response to satisfy subsequent requests without successful validation on the origin server.

This is often interpreted that by specifying max-age together with must-revalidate you can determine when a content is stale (after max-age seconds) and then it must validate at the origin server before it can serve the content.

In practice, however, for the reasons given above, it seems that must-revalidate always leads to a validation of each request on the origin server.

Answer 2

Try changing these lines

NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
NSURLSession *session = [NSURLSession sessionWithConfiguration:config];

to this:

NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
config.urlCache = sharedURLCache; // make sure sharedURLCache is accessible from here
NSURLSession *session = [NSURLSession sessionWithConfiguration:config];