Haproxy continue to route sessions to a backend marked as down

Question

I'm using HaProxy 1.5.0 in front of a 3-node MariaDB cluster. HaProxy checks with a custom query/xinet service that each DB node has a synced status. When for some reason the check fails (the node for instance gets desynced or becomes donor), the corresponding backend in haproxy is marked down, but I can still see active sessions on it in haproxy statistics console, and queries in DB process list (this is possible because MariaDB service is still up and accepts queries, even though the cluster status is not synced). I was wondering why HaProxy does not close active connections when a backend becomes down, and dispatch them to other active backends? I get this expected behaviour when the MariaDB service is fully stopped on a given node (no session possible). Is there a specific option to allow this? Option redispatch seemed promising but it applies when connections are closed (not in my case) and it's already active in my config. Thanks for your help.

Answer 1

Here are the settings we're using to get the same behavior:

default-server port 9200 [snip] on-marked-down shutdown-sessions

The on-marked-down shutdown-sessions option, that tells HAProxy to close all connections to the backend server when it is marked as down.

Of course, you can add it to every individual server if you're not using a default-server directive :)