Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cloud-front backed with Nginx (which proxies to S3) randomly missing already cached items?

I wish to serve images from a S3 bucket with Cloudfront as CDN frontend, for that I tried the following:

What I wish to acheive (Attempt 2) -- (Misses cloudfront cache randomly)

I have the following setup to serve images: ( Cloudfront --> Nginx --> S3 ) Cloudfront -> Nginx -> S3

<<<<<<<< Sample S3 headers >>>>>>>>>> Attempt-2 S3 headers

<<<<<<<< Sample Nginx -> S3 headers (Added Cache-Control) >>>>>>>>>> Attempt-2 Nginx->S3 headers

<<<<<<<< Sample Cloudfront -> Nginx -> S3 headers >>>>>>>>>> Attempt-2 Cloudfront->Nginx->S3 headers

What I am currently working with (Attempt 1) -- (Hits cloudfront as expected everytime)

Cloudfront -> S3

Cloudfront Settings:

  • Respects GET params to support urls like (http://cdn.example.com/abc.jpg?v=1)
  • Cache TTL set to 157680000 ( Fallback for Cache-Control )

What am I screwing up in Attemp-2 with my headers ? ( Cloudfront missing randomly )

Url(http://cdn.example.com/abc.jpg) & Url(http://cdn.example.com/abc.jpg?v=1) both will have same ETag, is that fine ?

Update

#AWS followed up on forums.aws.amazon.com, still waiting for a reply:

https://forums.aws.amazon.com/thread.jspa?threadID=144286&tstart=0#

Update2

  • Recent hit/miss behavioral change from cloudfront without changing anything.
  • Earlier the hits/misses were random with no fix pattern
  • Now, (with no change on my end) I am getting all hits 1 day and all misses the next day.
  • This suggests that its 24 hour cache but TTL and cache headers suggest 5 year cache expiry.
  • This is again weird and without any explanations.

Hey, AWS can you see this ???

like image 394
Yugal Jindle Avatar asked Jan 21 '14 04:01

Yugal Jindle


People also ask

What is CloudFront in S3?

CloudFront is a content delivery network (CDN) service that delivers static and dynamic web content, video streams, and APIs around the world, securely and at scale. By design, delivering data out of CloudFront can be more cost effective than delivering it from S3 directly to your users.

How do I know if CloudFront is caching?

To display CloudFront cache statisticsSign in to the AWS Management Console and open the CloudFront console at https://console.aws.amazon.com/cloudfront/v3/home . In the navigation pane, click Cache Statistics.

How long does CloudFront cache last?

By default, CloudFront caches a response from Amazon S3 for 24 hours (Default TTL of 86,400 seconds). If your request lands at an edge location that served the Amazon S3 response within 24 hours, then CloudFront uses the cached response.

How do I enable caching in CloudFront?

If you want OPTIONS responses to be cached, do the following: Choose the options for default cache behavior settings that enable caching for OPTIONS responses. Configure CloudFront to forward the following headers: Origin , Access-Control-Request-Headers , and Access-Control-Request-Method .


1 Answers

After about 4 months of repetitive to-n-fro with amazon support failed to resolve the issue.

All problems still persisting:

  • The cache expires in about a day and misses after 24 hours. ( My expiry is 1 year )

  • All headers and aws settings verified by amazon support themselves

  • Unfortunately, the company is still paying for this awful experience due to lockin.

enter image description here

------ After 24 hours ------

enter image description here

------ After 24 hours ------

enter image description here

------ After 24 hours ------

enter image description here

-------- And so on.. -------

Concluding, The problem still stands unresolved and amazon support seems to have given up. This is quiet a strange experience since aws is something we generally take for granted.

:(

like image 192
Yugal Jindle Avatar answered Sep 22 '22 05:09

Yugal Jindle