How can I get control of Google App Engine caching behavior in WebKit (etags gone crazy)?

Question

Situation: running a Google App Engine site with my static content's default_expiration set to "14d"

Problem: in Chrome and Safari, visiting a URL (not reloading, just putting the cursor in the address bar and hitting Enter), causes a ton of requests to be fired with If-None-Match headers. The responses are always 304 Not Modified, as expected. I can watch these requests get fired in a debugging proxy like Charles or Fiddler.

Want: to avoid these requests and 304 responses entirely for static content -- simply trust the browser's cached content when it's available.

We use the standard "cache static content for a really long time, we'll take care of appending ?version={version} modifications to our query strings when we need to bust the cache" system, so we'd really like to avoid the 304's.

Belief: I think this is caused by the etag header that app engine sends down with every static content response. The app engine SDK does not send this header down, and I don't see this 304 behavior when messing around with the SDK.

Any advice? Can you turn off etags for app engine's static content?

Updated with an example piece of static content: http://www.khanacademy.org/stylesheets/shared-package/compressed.css

Answer 1

While I don't believe there is any way to control the etags header behavior for GAE, this is caused by a bug in WebKit that causes all static content to be re-downloaded when receiving a 302 redirect after a POST request.

Once WebKit fixes this bug, the issue should go away.

If you must, you can temporarily work around this specific redirect-after-POST bug by redirecting via a Refresh header instead of using a 302 redirect.

https://bugs.webkit.org/show_bug.cgi?id=38690

WebKit image reload on Post/Redirect/Get

http://www.google.com/support/forum/p/Chrome/thread?tid=72bf3773f7e66d68&hl=en

Answer 2

You need to remove both the Last-Modified and ETag headers.

By removing the ETag header, you disable caches and browsers from being able to validate files, so they are forced to rely on your Cache-Control and Expires header. Entity tags (ETags) are a mechanism to check for a newer version of a cached file.

Removing both the Last-Modified and ETag header, you will totally eliminate If-Modified-Since and If-None-Match requests and their 304 Not Modified responses, so a file will stay cached without checking for updates until the Expires header indicates new content is available.

More info here: http://www.samaxes.com/2008/04/htaccess-gzip-and-cache-your-site-for-faster-loading-and-bandwidth-saving/.

Unfortunately I don't know how you can turn them off for GAE's static content.

Answer 3

Chrome 9.0, Windows. When loading your home page, default.css, as well as all other .css files are served from cache, without making a request. I think this is a browser-specific behavior, you need to check other browsers as well.

Also, check this Google instructions, they helped me a lot when tuning caching params: http://code.google.com/speed/page-speed/docs/caching.html

Answer 4

Since this is an issue with Chrome and Safari, you could use HTML5 App Cache to prevent server calls on static resources completely. Check out an example here.

Answer 5

Your ETag value is fine. ETag does not force revalidation. It just enables it to be more reliable then last modification date. I just browsed to your static content example using Chrome 9, and your content is cached and not unnecessarily revalidated. The problem you saw could be related to "always re-validate" browser settings, which are not the default for most browsers. It could also be a Mac webkit related bug.