ETags: Validation and Cache max-age

Question

I have been using ETags for a good while now and I am pleased with how they can be used and what they allow to be done.

However, I am sometimes experiencing ETag validation requests. On my server, I can see that my browser hits the resource URL, and then gets the "Not changed" response. But, it still is a request. 

How can a clever webserver (I am coding mine in NodeJS) avoid being hit with validation for as long as cache-control: max-age=N goes?

For instance, my max-age indicates a TTL of 30 days. The client shouldn't validate the Etag in that time, at all. Is that possible?

Answer 1

I see that problem intermittently on my end (FF45 under Ubuntu). Some files get checked on each reload, and most are not. I'm not too sure what would tell Firefox that such or such file should be checked on each load.

According to a post by jscher2000 on a blog, Firefox first checks the Last-Modified and if out of date, then sends a request; request which will include the value of the server ETag in an If-None-Match, and I can also see the If-Modified-Since header.

Generally speaking, Firefox does not re-request or revalidate cached files that have not yet expired. You can change Firefox's behavior on the client side by changing a setting in about:config (apparently you need to clear the cache and restart Firefox for the change to take effect):

browser.cache.check_doc_frequency (defaults to after expiration) browser.cache.check_doc_frequency not working? @ mozillaZine Forums (clear cache/restart)

It looks like my check_doc_frequency parameter is set to 3, meaning that it should check for the documents only if they are considered as being out of date.

Possible values and their effects

0 — Check for a new version of a page once per session (a session starts when the first application window opens and ends when the last application window closes).
1 — Check for a new version every time a page is loaded.
2 — Never check for a new version - always load the page from cache.
3 — Check for a new version when the page is out of date. (Default)

One thing that I do is that I use both of the following headers:

Cache-Control: max-age=3600
Expires: Sun, Mar 27 2016 21:13:50

There may be a confusion of some sort and if only max-age is defined, then you run in some odd case scenario and miss that specific case.

Just in case, there is my full header:

Cache-Control: max-age=3600,public
Connection: keep-alive, Keep-Alive
Date: Mon, 28 Mar 2016 02:56:20 GMT
Etag: 6b395ccb5b0a913f1828cce3e2756bdc
Expires: Mon, 28 Mar 2016 03:56:19 GMT
Keep-Alive: timeout=15, max=5
Server: Apache
Set-Cookie: ...

The duplicate Keep-Alive value in the Connection field comes from Apache.

Answer 2

Not possible to force this at server level as it's the client that decides this.

Cache control headers like ETags are hints not directives. The browser is free to ignore these hints (for example when you refresh a page you explicitly ask the browser to recheck the resources). Some browsers also periodically recheck resources.

Saying that, if your cache control headers are set up correctly then these requests should be infrequent and the norm should be for the requests not to be sent.