Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Selecting a backend database for Grafana

Graphite, Elastisearch, Cloudwatch, Prometheus, InfluxDB are all supported backends for Grafana. I am creating an application with grafana front-end, but an not being able understand how these backends differ and which would be the best to use for my application (would prefer open-source). My use case is a static log file being imported from an external server which I want to parse and fill-in the DB to be consumed by grafana. The data can have up to 5000 time-series data points for about a 100 measurement. The database need not be distributed. I would be glad to get some tips on how I can select a backing database out of these. Thanks in advance!!

like image 633
Aditya Kotwal Avatar asked Jun 07 '16 23:06

Aditya Kotwal


People also ask

How do I add a data source to Grafana?

Refer to Add a data source for instructions on how to add a data source to Grafana. Only users with the organization admin role can add data sources. Each data source has a specific Query Editor that is customized for the features and capabilities that the particular data source exposes.

What is a backend Grafana plugin?

Grafana’s backend plugin system exposes a couple of different capabilities, or building blocks, that a backend plugin can implement: The query data capability allows a backend plugin to handle data source queries that are submitted from a dashboard, Explore or Grafana Alerting.

Can I use a MySQL database for data visualization in Grafana?

Jack Wallen walks you through the process of using a MySQL database as a source for data visualization in Grafana. Grafana is one of the most widely-used interactive data visualization tools on the market. It’s open-source, powerful, highly configurable and free to use.

How do I set up a Grafana dashboard?

Configuring your data source is the first step to setting up your Grafana dashboard. Your data source could be a database or a collection of logs. Grafana supports pretty much every data source you can think of, from traditional databases like MySQL to log aggregation platforms like AWS Cloudwatch.


2 Answers

Good answer by Brian, but adding more. You have to think about monitoring as 3 sets of data, which unfortunately in OSS you need a large mix of tools and projects. The fundamentals of monitoring consist of metrics (numbers such as what Grafana is good at visualizing), events (unstructured text such as what ELK is good at collecting and visualizing), and metadata (relationships, configuration, and other elements which span the other two categories).

Most people will use different technology stacks for each.

Metrics:

  • Graphite - Old, but well proven (uses RRD data stores)
  • InfluxDB - Newest, but less proven. Probably the best technology today
  • Prometheus - Uses a proprietary binary file based data store.

Events:

  • ElasticSearch - Java based unstructured data store, needs a lot of hardware to scale.

Once you have the metrics and events to visualize you'll need a bunch of tools. On ElasicSearch the ELK stack is most common E = ElasticSearch L = Logstash (ingesting logs) K = Kibana (visualization). Another alternative is Greylog which is better than Kibana IMHO.

Grafana is common, but not the best visualization. Unfortunately, the OSS tools out there just aren't great with metrics today.

like image 160
Jonah Kowall Avatar answered Sep 18 '22 16:09

Jonah Kowall


That sounds like an event logging use case, so Elasticsearch is probably your best bet.

For metrics uses cases Prometheus would be a good choice.

like image 27
brian-brazil Avatar answered Sep 18 '22 16:09

brian-brazil