Varnish automagically adding load balancer IP to X-Forwarded-For header

Question

My request flow is as follows;

HAProxy --> Varnish (4.0.1) --> Apache web backends

When a new request comes in to HAProxy, the client's IP address is being added to the X-Forwarded-For header (which is good!). However, it looks like Varnish is adding the HAProxy IP as well. When the request gets to my vcl_recv routine, the X-Forwarded-For header is:

X-Forwarded-For: end-user-ip, haproxy-ip

You can see that in the varnishlog output:

*   << Request  >> 8
-   Begin          req 7 rxreq
-   Timestamp      Start: 1409262358.542659 0.000000 0.000000
-   Timestamp      Req: 1409262358.542659 0.000000 0.000000
-   ReqStart       192.168.1.103 48193
-   ReqMethod      PURGE
-   ReqURL         /some/path
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Authorization: Basic xxx
-   ReqHeader      User-Agent: curl/7.30.0
-   ReqHeader      Host: example.com
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: 1.2.3.4
-   ReqHeader      Connection: close
-   ReqUnset       X-Forwarded-For: 1.2.3.4
-   ReqHeader      X-Forwarded-For: 1.2.3.4, 192.168.1.101
-   VCL_call       RECV
-   ReqUnset       X-Forwarded-For: 1.2.3.4, 192.168.1.101
-   VCL_acl        NO_MATCH purge_acl
-   Debug          "VCL_error(403, Not allowed.)"
-   VCL_return     synth

The reason I need the accurate client IP address is so I can check it against ACL rules for PURGE/BAN. Since the last IP in the X-Forwarded-For header is that of HAProxy, the ACL check fails for all IPs. Here is the relevant section of my config:

acl purge_acl {
    "1.2.3.4";
}

sub vcl_recv {

    set req.backend_hint = load_balancer.backend();

    if (req.method == "PURGE") {
        if (!std.ip(req.http.X-forwarded-for, "0.0.0.0") ~ purge_acl) {
            return(synth(403, "Not allowed."));
        }
        ban("obj.http.x-url ~ " + req.url);
        return(synth(200, "Ban added"));
    }

}

Any ideas how I can rely solely on the X-Forwarded-For header from HAProxy, without Varnish tampering with it?

A side note, it seems that Varnish is doing exactly this (although this IS NOT in mv VCL config):

if (req.restarts == 0) {
    if (req.http.X-Forwarded-For) {
        set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
    } else {
        set req.http.X-Forwarded-For = client.ip;
    }
}

Answer 1

Varnish appends its default logic to any functions you define such as vcl_recv rather than purely overriding it. The default vcl_recv logic contains:

if (req.http.x-forwarded-for) {
    set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
    set req.http.X-Forwarded-For = client.ip;
}

as you noticed. What seems strange to me is that it seems like the Varnish default logic in vcl_recv is executing ahead of your vcl_recv logic. What value do you see for X-Forwarded-For within vcl_deliver if you define your own?

One thing you could do is parse out the first IP address like this where necessary:

set req.http.X-Forwarded-For = regsub(req.http.X-Forwarded-For, "^([^,]+),?.*$", "\1");