Nginx bypass cache if upstream is up and use cache if down

Question

To bypass cache if upstream is up (max-age 1) and use cache if down (proxy_cache_use_stale) I created following config:

proxy_cache_path   /app/cache/ui levels=1:2 keys_zone=ui:10m max_size=1g inactive=30d;
server {
    ...
    location /app/ui/config.json {
        proxy_cache ui;
        proxy_cache_valid 1d;
        proxy_ignore_headers Expires;           
        proxy_hide_header Expires;
        proxy_hide_header Cache-Control;
        add_header Cache-Control "max-age=1, public";
        proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
        add_header X-Cache-Status $upstream_cache_status;
        add_header X-Cache-Date $upstream_http_date;
        proxy_pass http://app/config.json;
    }
}

But cache is not used when upstream is down and client only gets 504 Gateway Timeout. I've already read following articles:

https://nginx.org/ru/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale

How to configure NginX to serve Cached Content only when Backend is down (5xx Resp. Codes)?

https://serverfault.com/questions/752838/nginx-use-proxy-cache-if-backend-is-down

And It does not work as I expect. Any help is appreciated.

Answer 1

Let's discuss a really simple setup with two servers. One running apache2 serving a simple html page. The other running nginx that reverse proxies to the first one.

http {
[...]

  proxy_cache_path /var/lib/nginx/tmp/proxy levels=2:2 keys_zone=one:10m inactive=48h max_size=16g use_temp_path=off;

  upstream backend {
    server foo.com;
  }

  server {
  [...]
    location / {
      proxy_cache           one;
      proxy_cache_valid     200 1s;
      proxy_cache_lock      on;

      proxy_connect_timeout 1s;
      proxy_cache_use_stale error timeout updating http_502 http_503 http_504;

      proxy_pass http://backend/
    }
  }
}

This setup works for me. The most important difference is the proxy_cache_valid 200 1s; It means that only responses with http code 200 will be cached, and will only be valid for 1 second. Which does imply that the first request to a certain resource will be get from the backend and put in the cache. Any further request to that same resource will be served from the cache for a full second. After that the first request will go to the backend again, etc, etc.

The proxy_cache_use_stale is the important part in your scenario. It basically says in which cases it should still serve the cached version although the time specified by proxy_cache_valid has already passed. So here you have to decided in which cases you still want to serve from cache.

The directive's parameters are the same as for proxy_next_upstream.

You will need these:

error: In case the server is still up, but not responding, or is not responding correctly.

timeout: connecting to the server, requesting or response times out. This is also why you want to set proxy_connect_timeout to something low. The default is 60s and is way to long for an end-user.

updating: there is already a request for new content on it's way. (not really needed but better from a performance point of view.)

The http_xxx parameters are not going to do much for you, when that backend server is down you will never get a response with any of these codes anyway.

In my real life case however the backend server is also nginx which proxies to different ports on the localhost. So when nginx is running fine, but any of those backends is down the parameters http_502, http_503 and http_504 are quit useful, as these are exactly the http codes I will receive.

The http_403, http_404 and http_500 I would not want to serve from cache. When a file is forbidden (403) or no longer on the backend (404) or when a script goes wrong (500) there is a reason for that. But that is my take on it.

Answer 2

This, like the other similar questions linked to, are examples of the XY Problem.

A users wants to do X, wrongly believes the solution is Y but cannot do Y and so asks for help on how to do Y instead of actually asking about X. This invariably results in problems for those trying to give an answer.

In this case, the actual problem, X, appears to be that you will like to have a failover for your backend but would like to avoid spending money on a separate server instance and would like to know what options are available.

The idea of using a cache for this is not completely off but you have to approach and set the cache like a failover server which means it has to be a totally separate and independent system from the backend. This rules out proxy_cache which is intimately linked to the backend.

In your shoes, I will set up a memcached server and configure this to cache your stuff but not ordinarily serve your requests except on a 50x error.

There is a memcached module that comes with Nginx that can be compiled and used but it does not have a facility to add items to memcached. You will have to do this outside Nginx (usually in your backend application).

A guide to setting memcached up can be found here or just do a web search. Once this is up and running, this will work for you on the Nginx side:

server {
    location / {
        # You will need to add items to memcached yourself here
        proxy_pass             http://backend;
        proxy_intercept_errors on
        error_page             502 504 = @failover;
    }

    location @failover {
        # Assumes memcached is running on Port 11211
        set            $memcached_key "$uri?$args";
        memcached_pass host:11211;
    }
}      

Far better than the limited standard memcached module is the 3rd party memc module from OpenResty which allows you to add stuff directly in Nginx.

OpenResty also has the very flexible lua-resty-memcached which is actually the best option.

For both instances, you will need to compile them into your Nginx and familiarise yourself on how to set them up. If you need help with this, ask a new question here with the OpenResty tag or try the OpenResty support system.

Summary

1. What you actually need is a failover server.
2. This has to be separate and independent of the backend.
3. You can use a caching system as this but it cannot be proxy_cacheif you cannot live with getting cached results for the minimum time of 1 second.
4. You will need to extend a typical Nginx installation to do this.