Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Increasing Prometheus storage retention

Tags:

Hi I have Prometheus server installed on my AWS instance but the data is been removed automatically after 15 days. I need to have data for an year or months, is there anything I need to change in my prometheus configuration? Or do I need any extensions like Thanos, I am new to Prometheus so please be easy on the answers

like image 526
Rohit Bharati Avatar asked Dec 12 '19 06:12

Rohit Bharati


People also ask

How long is data stored in Prometheus?

By default the retention is configured to 15 days. The amounts of data stored on disk depends on retention — higher retention means more data on disk. The lowest supported retention in Prometheus is 2 hours (2h).

How much data can Prometheus handle?

Prometheus can handle millions of time series. However, with the above mentioned default setting for storage. local. target-heap-size , you will be limited to about 200,000 time series simultaneously present in memory.

Where does Prometheus store all its metric data?

As you can check the Prometheus Dockerfile on Github (https://github.com/prometheus/prometheus/blob/master/Dockerfile#L24), the working directory is /prometheus and that where you will find all the metrics and data. And this is the *Time Series Database * which you can't decode .


2 Answers

  1. Edit the prometheus.service file

vi /etc/systemd/system/prometheus.service

  1. add "--storage.tsdb.retention.time=1y" below to "ExecStart=/usr/local/bin/prometheus \" line.

So the config will look like bellow for 1 year of data retention.

[Unit] Description=Prometheus Wants=network-online.target After=network-online.target  [Service] User=prometheus Group=prometheus Type=simple ExecStart=/usr/local/bin/prometheus \     --config.file /etc/prometheus/prometheus.yml \     --storage.tsdb.path /var/lib/prometheus/ \     --web.console.templates=/etc/prometheus/consoles \     --web.console.libraries=/etc/prometheus/console_libraries \     --web.external-url=http://34.89.26.156:9090 \     --storage.tsdb.retention.time=1y [Install] WantedBy=multi-user.target 
like image 51
Prabhu Avatar answered Oct 03 '22 20:10

Prabhu


There's the --storage.tsdb.retention.time flag that you can set when you start Prometheus. It defines how long data is kept in the time-series database (TSDB). The default is 15 days.

So, to increase the retention time to a year, you should be able to set this to something like:

--storage.tsdb.retention.time=1y # or --storage.tsdb.retention.time=365d 

See the Prometheus documentation.

like image 34
weibeld Avatar answered Oct 03 '22 18:10

weibeld