NGINX read body from proxy_pass response

Question

I have two servers:

1. NGINX (it exchanges file id to file path)
2. Golang (it accepts file id and return it's path)

Ex: When browser client makes request to https://example.com/file?id=123, NGINX should proxy this request to Golang server https://go.example.com/getpath?file_id=123, which will return the response to NGINX:

{
  data: {
    filePath: "/static/..."
  },
  status: "ok"
}

Then NGINX should get value from filePath and return file from the location.

So the question is how to read response (get filePath) in NGINX?

Answer 1

I assume you are software developer and your have full control over your application so there is no need to force square peg in a round hole here.

Different kinds of reverse proxies support ESI(Edge Side Includes) technology which allow developer to replace different parts of responce body with content of static files or with response bodies from upstream servers.

Nginx has such technology as well. It is called SSI (Server Side Includes).

location /file {
    ssi on;
    proxy_pass http://go.example.com;
}

Your upstream server can produce body with content <!--# include file="/path-to-static-files/some-static-file.ext" --> and nginx will replace this in-body directive with content of the file.

But you mentioned streaming...

It means that files will be of arbitrary sizes and building response with SSI would certainly eat precious RAM resources so we need a Plan #B.

There is "good enough" method to feed big files to the clients without showing static location of the file to the client. You can use nginx's error handler to server static files based on information supplied by upstream server. Upstream server for example can send back redirect 302 with Location header field containing real file path to the file. This response does not reach the client and is feed into error handler.

Here is an example of config:

location /file {
    error_page 302 = @service_static_file;
    proxy_intercept_errors on;
    proxy_set_header Host            $host;
    proxy_pass http://go.example.com;
}

location @service_static_file {
    root /hidden-files;
    try_files $upstream_http_location 404.html;
}

With this method you will be able to serve files without over-loading your system while having control over whom do you give the file.

For this to work your upstream server should respond with status 302 and with typical "Location:" field and nginx will use location content to find the file in the "new" root for static files.

The reason for this method to be of "good enough" type (instead of perfect) because it does not support partial requests (i.e. Range: bytes ...)