fallback for an nginx upstream proxy

Question

I have an nginx instance in front of a cluster of image servers:

  upstream img-farm-1  {
    server 10.0.1.1;
    server 10.0.1.2;
    server 10.0.1.3;
    server 10.0.1.4;
    # etc
  }

  location ~ ^/static: {
    rewrite /static:(.*) /$1 break;
    proxy_pass http://img-farm-1;
    limit_except GET {
        allow all;
    }
  }

This cluster is being superceded by a new cluster that is coming on line, and for a while, I want to serve the image from the old cluster, but fallback on the new cluster if the image is new or has been migrated from the old to the new. Once the migration is complete I can go back to the original setup.

So I thought I could do

  upstream img-farm-2  {
    server 10.0.2.1;
    server 10.0.2.2;
    server 10.0.2.3;
    server 10.0.2.4;
    server 10.0.2.5;
    # etc
  }

  location ~ ^/static: {
    access_log /var/log/nginx/static.access.log;
    rewrite    /static:(.*) /$1 break;
    proxy_pass http://img-farm-1;
    error_page 404 = @fallback-2;
  }

  location @fallback-2 {
    access_log /var/log/nginx/static-2.access.log;
    proxy_pass http://img-farm-2;
  }

But this doesn't work. I'm seeing 404s in static.access.log but the error_page 404 directive is not being acted upon, insofar as there's nothing at all being written to static-2.access.log.

I'm pretty sure I can't use try_files because, well duh, there aren't any local files, everything is proxified.

Has anyone done something like this before? What am I missing?

Answer 1

Silly me. All that was needed was proxy_intercept_errors on; in the first location

Answer 2

I had a similar situation but I want to fallback onto another server if my first server is down (status) 502. I got it working (nginx/1.17.8) without needing proxy_intercept_errors`.

Also, for anyone using proxy_pass with a URI (/ in this example) you need to use a slightly different configuration or you will get and error (see below).

location /texts/ {
  proxy_pass http://127.0.0.1:8084/;
  proxy_set_header X-Forwarded-For $remote_addr;
  error_page 502 = @fallback;
}

location @fallback {
  # This will pass failed requests to /texts/foo above to
  #  http://someotherserver:8080/texts/foo
  proxy_pass http://someotherserver:8080$request_uri;
}

For whatever reason, Nginx does not allow a slash here:

location @fallback {
  proxy_pass http://someotherserver:8080/;
}

And will throw:

nginx: [emerg] "proxy_pass" cannot have URI part in location given by regular expression, or inside named location, or inside "if" statement, or inside "limit_except" block in

Unfortunately without the / you cannot (AFAIK) proxy_pass a subdirectory. No RegEx with $1 also does not work because it doesn't support spaces.