Prometheus timestamps

Question

I wrote an exporter which exposes timestamp next to the metric like:

test_load_min{app="web01"} 1 1483228810000
test_load_min{app="db01"} 2 1483228820000
test_load_min{app="email01"} 3 1483228830000

According to https://prometheus.io/docs/instrumenting/exposition_formats/ this should be fine for Prometheus, but querying test_load_min in Prometheus UI returns empty result. Same without timestamps works fine. Anyone has an idea what's wrong there?

Answer 1

The timestamp 1483228810000 converts back to January of this year.

Prometheus omits time series from query results if the timestamp for which the query is executed is more than 5 minutes away from the nearest sample.

It is also not advised to use timestamps in this way.

Answer 2

Timestamps are not designed for the uploading of historical data.

There's a plan to add support for bulk historical uploads, tracked here. The issue is still open for now though.

In the meantime, you could look at promqueen which claims to solve this problem (though not for influxdb).