Prometheus: how to drop a target based on Consul tags

Question

My Prometheus server gets its list of targets (or "services", in Consul's lingo) from Consul. I only want to monitor a subset of these targets. This should be possible via Prometheus's regex mechanism, but the correct configuration eludes me. How is this done?

Answer 1

I've scoured the web and there is not a single example showing how its done, so for posterity - the following configuration will drop all consul services marked with the 'ignore-at-prometheus' tag

# ignore consul services with 'ignore_at_prometheus' tag
# https://www.robustperception.io/little-things-matter/
relabel_configs:
- source_labels: ['__meta_consul_tags']
  regex: '(.*),ignore-at-prometheus,(.*)'
  action: drop