Nginx real client IP to TCP stream backend

Question

I'm trying to use Nginx as proxy for my TCP daemon, to make Nginx be SSL/TLS frontend, as well as load control.

My backend app needs real client IP, and this is a problem.

stream {
    server {
        listen     3333;
        proxy_pass 127.0.0.1:2222;
    }
}

I found one solution in docs:

proxy_bind $remote_addr transparent;

But it's too complicated:

"In order for this parameter to work, it is necessary to run nginx worker processes with the superuser privileges and configure kernel routing table to intercept network traffic from the proxied server."

Is there any other way to pass $remote_addr to backend?

I tried modify client message body using sub_filter, send http headers, etc. But all of this can work only with http context, not with stream unfortunately.

Answer 1

To my knowledge, there are only two solutions: proxy_bind and proxy_protocol.

proxy_bind

As you quoted from the documentation, the work processes will need superuser privileges. Obviously. this not a best practice. Additionally, it might cause connection issues back to the remote clients.

Assuming the Nginx is running as the nginx user, run this command to give it permissions.

usermod -aG sudo nginx

proxy_protocol

This solution requires the upsteam destination (e.g. the backend app) accepts PROXY protocol.

stream {
    server {
        listen         3333;
        proxy_pass     127.0.0.1:2222;
        proxy_protocol on;
    }
}

The above solutions assume the Nginx server is the entry point to the network. If there is a edge device (e.g. load balancer), it is very likely it is changing the source IP. In this case, you will need to enable proxy protocol on the edge device and enable proxy_protocol listener in the server block. I haven't tested it, but something like this should work.

stream {
    server {
        listen            3333 proxy_protocol;
        proxy_pass        127.0.0.1:2222;
        proxy_protocol    on;
        set_real_ip_from  $proxy_protocol_addr;
    }
}

Answer 2

My setup:

1. Nginx stream module as Tcp proxy for ssh and https
2. Nginx http module to serve my content

My requirement: Having the real client ip inside access.log (instead of 127.0.0.1 from stream module) and also for geoblocking

My solution:

1. Activate proxy_protocol (to add the additional information in the requests)
2. add "proxy_procotol" in http listen directive (for the http server to accept)
3. add extra proxy between stream module and ssh to remove proxy_protocol for ssh being able to read
4. replace "$remote_addr" with "$proxy_protocol_addr" in access.log

For geoblocking you also have to use proxy_protocol_addr, but i'll omit describing this here to keep it short.

nginx.conf:

...
stream {

    upstream ssh {
        server 127.0.0.1:2222;
    }

    upstream https {
        server 127.0.0.1:444;
    }

    map $ssl_preread_protocol $upstream {
        default ssh;
        "TLSv1.2" https;
        "TLSv1.3" https;
        "TLSv1.1" https;
        "TLSv1.0" https;
    }

    server {
        listen 443;
        proxy_pass $upstream;
        proxy_protocol on;
        ssl_preread on;
    }

    server {
        listen 2222 proxy_protocol;
        proxy_pass 192.168.2.76:22;
    }
}

http {
    log_format  main  '$proxy_protocol_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    ...
    server {
        listen 444 ssl proxy_protocol;
    ... 
    }
}