maxconn limit per backend in haproxy

Question

Our haproxy loadbalancer opens thousands of connections to its backends even though its settings say to open no more than 10 connections per server instance (see below). When I uncomment "option http-server-close" the number of backend connection drops however I would like to have keep-alive backend connections.

Why maxconn is not respected with http-keep-alive? I verified with ss that the opened backend connections are in ESTABLISHED state.

defaults
     log global
     mode    http
     option http-keep-alive
     timeout http-keep-alive 60000
     timeout connect 6000
     timeout client  60000
     timeout server  20000


 frontend http_proxy
     bind    *:80
     default_backend backends

 backend backends
     option prefer-last-server

     # option http-server-close
     timeout http-keep-alive 1000
     server s1 10.0.0.21:8080 maxconn 10
     server s2 10.0.0.7:8080  maxconn 10
     server s3 10.0.0.22:8080 maxconn 10
     server s4 10.0.0.16:8080 maxconn 10

Answer 1

In keep-alive mode idle connections are not accounted. As explained in this HAProxy mailthread

The thing is, you don't want to leave requests waiting in a server's queue while the server has a ton of idle connections.

This even makes more sense, knowing that browsers initiate preconnect to improve page performance. So in keep-alive mode only outstanding/active connections are taken into account.

You can still enforce maxconn limits regardless of the connection state using tcp mode, especially that I don't see a particular reason to using mode http in your current configuration (apart from having reacher logs).
Or you can use http-reuse with http mode to achieve a lowest number of concurrent connections.