Replace "exported_instance" label value from Prometheus

Question

this label (exported_instance) is always added in every metric automatically.

myMetric{exported_instance="HP-probook", instance="ss:9966", job="ss"}

How can I replace during scrape time from prometheus.yml? Is it possibile? Or I need to add new one?

relabel_configs:
  - source_labels: [__address__]
   regex: 'HP-probook'
    target_label: new_label
    replacement: 'HP-probook-2'

I understand that it is added automatically and for this reason it is not clear to me if I can replace it without adding another one.

Thanks Alen

Answer 1

job and instance labels are somewhat special with Prometheus. instance comes from the __address__ label and represents a host and port from which the metric came.

You get metrics with exported_ prefix if the collected data already have one of the predefined labels. In other words, exported_instance is the instance label in your exporter.

Solution 1

The best way around this is to avoid using any of the special labels in the exporter.

Solution 2

Configure Prometheus scraping job to keep you exporter label value instead of the value Prometheus would normally use, using honor_labels

Solution 3

If for some reason it is mandatory to use one of these labels, you can perform relabeling at the metric_relabel_configs:

metric_relabel_configs:
# copy exported_instance value to label foo
- source_labels: [exported_instance]
  target_label: foo
# remove exported_instance label
- action: labeldrop
  regex: ^exported_instance$

Remember that you cannot use relabel_configs to deal with labels that come from an exporter, you need metric_relabel_configs for this. Here is another my answer with the explanation: prometheus relabel_config drop action not working