The behavior of varnish on MISS

Question

Consider this scenario: The Varnish cache has a MISS and the backend server is now regenerating the requested content. During the generation time a second request comes in and also gets an MISS. Does varnish send this request to the backend while the other request is pending? What if thousand requests come in between this time. Server would crash right? Every request would make it slower.

Is this correct or is varnish "synchronizing" those scenarios to prevent such a problem?

Thank you in advance!

Answer 1

Varnish sends all the requests to the backend. I.e. it does not queue other requests and issue just one backend request and use its response for all.

However Varnish has a grace option that lets you keep old, expired content in cache for these types of situations.

For example consider the following VCL:

sub vcl_recv {
  if (req.backend.healthy) {
    set req.grace = 5m;
  } else {
    set req.grace = 24h;
  }
}

sub vcl_fetch {
   set beresp.grace = 24h;
}

Now if a backend is healthy (see backend polling) and a request results in a MISS, the first request is sent to a backend. If another request comes for the same content, but there is an item in the cache with age <TTL+req.grace (in this case 5 minutes), that request will get the "stale" content instead. This happens as long as either the first request that resulted in a MISS gets a response from the backend (and the cache is fresh again) or the age of the item becomes greater than TTL+req.grace.

If the backend was down (req.backend.healthy == FALSE), stale content would be served as long as age<TTL+24h.

You might also want to check out the Saving a request section of the Varnish book for a more thorough example and an exercise.

Fixed: unescaped < character.

Fixed more: There was another unescaped < character...