Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

UDP forwarding with nginx

I have a main syslog server that is receiving syslog from several sources, and I want to send those logs to a Graylog cluster. To help the cluster keep up (on some slow VMs), I need to be able to load balance the messages to Graylog, as sometimes they come in massive chunks from the endpoints (some send 5k logs in bursts every 10 seconds).

I'm trying to use nginx as a load balancer for the syslog messages, but I can't seem to get it to work, and it seems to be because nginx is looking for responses from the Graylog servers. With UDP, it's not going to get a response. At least this is what I think is happening.

The error I'm getting is this:

2016/12/01 11:27:59 [error] 2816#2816: *210325 no live upstreams while connecting 
  to upstream, udp client: 10.0.1.1, server: 0.0.0.0:11016, 
  upstream: "juniper_close_stream_backend", bytes from/to client:932/0, 
  bytes from/to upstream:0/0

As an example of this rule in my nginx.conf, it looks like:

stream {
    server {
        listen 11016 udp;
        proxy_pass juniper_close_stream_backend;
    }
    upstream juniper_close_stream_backend {
        server 10.0.1.2:11016;
        server 10.0.1.3:11016;
        server 10.0.1.4:11016;
    }
}

In this instance, my syslog box is 10.0.1.1, and my downstream Graylog boxes are 10.0.1.[2-4]. I see this error message for all of them.

Any clue on what is happening? When I run tcpdump on the Graylog boxes, I'm seeing the traffic coming from the load balancer, which means it's working. But I think nginx is expecting a response and is giving me an error.

like image 382
jasonmclose Avatar asked Dec 01 '16 17:12

jasonmclose


1 Answers

So this did seem to be the solution (in my note above).

If using my example from above, you want this to look like:

 stream {
     server {
         listen 11016 udp;
         proxy_pass juniper_close_stream_backend;
         proxy_responses 0;
     }
 }

This tells nginx not to expect a response, which it shouldn't need from UDP. I don't know why their examples don't show this when discussing DNS, which can be entirely UDP driven.

like image 165
jasonmclose Avatar answered Oct 22 '22 03:10

jasonmclose