How to tell cloudfront to not cache 302 responses from S3 redirects, or, how else to workaround this image caching generation issue

Question

I'm using Imagine via the LIIPImagineBundle for Symfony2 to create cached versions of images stored in S3.

Cached images are stored in an S3 web enabled bucket served by CloudFront. However, the default LIIPImagineBundle implementation of S3 is far too slow for me (checking if the file exists on S3 then creating a URL either to the cached file or to the resolve functionality), so I've worked out my own workflow:

1. Pass client the cloudfront URL where the cached image should exist
2. Client requests the image via the cloudfront URL, if it does not exist then the S3 bucket has a redirect rule which 302 redirects the user to an imagine webserver path which generates the cached version of the file and saves it to the appropriate location on S3
3. The webserve 301 redirects the user back to the cloudfront URL where the image is now stored and the client is served the image.

This is working fine as long as I don't use cloudfront. The problem appears to be that cloudfront is caching the 302 redirect response (even though the http spec states that they shouldn't). Thus, if I use cloudfront, the client is sent in an endless redirect loop back and forth from webserver to cloudfront, and every subsequent request to the file still redirects to the webserver even after the file has been generated.

If I use S3 directly instead of cloudfront there are no issues and this solution is solid.

According to Amazon's documentation S3 redirect rules don't allow me to specify custom headers (to set cache-control headers or the like), and I don't believe that CloudFront allows me to control the caching of redirects (if they do it's well hidden). CloudFront's invalidation options are so limited that I don't think they will work (can only invalidate 3 objects at any time)...I could pass an argument back to cloudfront on the first redirect (from the Imagine webserver) to fix the endless redirect (eg image.jpg?1), but subsequent requests to the same object will still 302 to the webserver then 301 back to cloudfront even though it exists. I feel like there should be an elegant solution to this problem but it's eluding me. Any help would be appreciated!!

Answer 1

I'm solving this same issue by setting the "Default TTL" in CloudFront "Cache Behavior" settings to 0, but still allowing my resized images to be cached by setting the CacheControl MetaData on the S3 file with max-age=12313213.

This way redirects will not be cached (default TTL behavior) but my resized images will be (CacheControl max-age on s3 cache hit).

Answer 2

If you really need to use CloudFront here, the only thing I can think of is that you don’t directly subject the user to the 302, 301 dance. Could you introduce some sort of proxy script / page to front S3 and that whole process? (or does that then defeat the point).

So a cache miss would look like this:

• Visitor requests proxy page through Cloudfront.
• Proxy page requests image from S3
• Proxy page receives 302 from S3, follows this to Imagine web server
• Ideally just return the image from here (while letting it update S3), or follow 301 back to S3
• Proxy page returns image to visitor
• Image is cached by Cloudfront