How to ping targets using blackbox_exporter with prometheus

Question

I'm trying to ping a list of targets using blackbox_exporter with prometheus but I seem to only be able to probe blackbox_exporters and not the actual targets I want to check.

I can't find any documentation in blackbox on where to list the targets should be listed so I made the odd assumption that it uses the targets provided in the prometheus config but from what I understand that just makes prometheus believe that there are many blackboxes to probe.

This is my blackbox_exporter config

 modules:
  icmp:
    prober: icmp
    timeout: 5s
    icmp:
      preferred_ip_protocol: ip4

However when I visit the web GUI for blackbox the config contains a bunch of parameters which I didn't specify.

modules:
  icmp:
    prober: icmp
    timeout: 5s
    http:
      ip_protocol_fallback: true
    tcp:
      ip_protocol_fallback: true
    icmp:
      preferred_ip_protocol: ip4
      ip_protocol_fallback: true
    dns:
      ip_protocol_fallback: true

And this is my prometheus config

global:
  scrape_interval:     15s
  evaluation_interval: 15s

  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [icmp]
    static_configs:
      - targets:
        - icmp-target1 # supposed to be a switch, router, pc or anything that responds to ping
        - icmp-target2

Expected result: Somewhere along the journey I expected my targets to be pinged by blackbox and prometheus collecting the result.

Actual result: Prometheus sends a probe request through HTTP for every target listed in its target list.

Answer 1

Although slightly confusing, the blackbox_exporter README does explain how to configure it, see section Prometheus Configuration.

Your blackbox configuration is correct.

For your Prometheus configuration, you need something like the following. I am assuming that the blackbox exporter and Prometheus are colocated (hence the localhost), otherwise adapt.

# this is to scrape blackbox itself (this is optional)
- job_name: blackbox
  static_configs:
  - targets: ['localhost:9115']


- job_name: blackbox-ping
  metrics_path: /probe
  params:
    module: [icmp]
  static_configs:
    - targets:
      - 192.168.1.1   # <== Put here your targets
  relabel_configs:    # <== This comes from the blackbox exporter README
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: localhost:9115 # Blackbox exporter.

The other confusing part is answering the question: "How do I use the blackbox exporter for multiple protocols, say ICMP and HTTP?" In this case, there are various options, but the clearer one is to have one section per protocol. This is why I call the ICMP probes blackbox-ping. Say we wanted to have also HTTP probes, we would add another section:

- job_name: blackbox-http
  metrics_path: /probe
  params:
    module: [http_2xx]
  static_configs:
    - targets:
      - https://www.google.com  # <== your targets here
  relabel_configs:              # <== This comes from the blackbox exporter README
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: localhost:9115 # Blackbox exporter.

You would also need the corresponding blackbox configuration:

modules:
  http_2xx:        # <== This is the new section for HTTP
    prober: http
    timeout: 10s   # <== This depends on what you want to do
    http:
      valid_status_codes: []  # Defaults to 2xx
      method: HEAD              # <== This depends on what you want to do
      no_follow_redirects: true # <== this depends on what you want to do
  icmp:                         # <== this is the one you already have
    prober: icmp
    timeout: 10s                # <== This depends on what you want to do
    icmp:
      preferred_ip_protocol: ip4