How to persist data in Prometheus running in a Docker container?

Question

I'm developing something that needs Prometheus to persist its data between restarts. Having followed the instructions

$ docker volume create a-new-volume $ docker run \     --publish 9090:9090 \     --volume a-new-volume:/prometheus-data \     --volume "$(pwd)"/prometheus.yml:/etc/prometheus/prometheus.yml \     prom/prometheus 

I have a valid prometheus.yml in the right directory on the host machine and it's being read by Prometheus from within the container. I'm just scraping a couple of HTTP endpoints for testing purposes at the moment.

But when I restart the container it's empty, no data from the previous run. What am I missing from my docker run ... command to persist the data into the a-new-volume volume?

Answer 1

Use the default data dir, which is /prometheus. To do that, use this line instead of what you have in your command:

... --volume a-new-volume:/prometheus \ ... 

Found here: https://github.com/prometheus/prometheus/blob/master/Dockerfile

Surprisingly is not mentioned in the image docs

Answer 2

I had the same issue a today, but I was using a docker composer file. So wrapping up all what was in comments of other answers and what worked for me. In case setting up the Prometheus docker via yaml compose file...

First create a folder for the volume on the host machine, e.g.:

$ mkdir /tmp/prometheus 

Then change the folder owner to nobody, like (use sudo if needed):

$ chown 65534:65534 /tmp/prometheus 

Then add volume to the yaml configuration file:

prometheus:   image: prom/prometheus   container_name: prometheus   ports:     - 9090:9090   volumes:     - /tmp/prometheus:/prometheus     - ./prometheus.yml:/etc/prometheus/prometheus.yml 

That should do it.