Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Stackdriver vs ELK for app engine

Im a little confused about this because the docs say I can use stackdriver for "Request logs and application logs for App Engine applications" so does that mean like web requests? Does that mean like millions of web requests?

Stackdriver's pricing is per resource so does that mean I can log all of my web servers web request logs (which would be HUGE) for no extra cost (meaning I would not be charged by the volume of storage the logs use)?

Does stackdriver use GCP cloud storage as a backend and do I have to pay for the storage? It just looks like I can get hundreds of gigabytes of log aggregation for virtually no money just want to make sure Im understanding this.

I bring up ELK because elastic just partnered with google so it must not do everything elasticsearch does (for almost no money) otherwise it would be a competitor?

like image 923
red888 Avatar asked May 01 '17 12:05

red888


People also ask

Is Elk stack still free?

High Cost of OwnershipELK stack software is free to use, but building, growing, and maintaining the ELK stack requires infrastructure and resources.

What can elk stack be used for?

Often referred to as Elasticsearch, the ELK stack gives you the ability to aggregate logs from all your systems and applications, analyze these logs, and create visualizations for application and infrastructure monitoring, faster troubleshooting, security analytics, and more.

What three open source applications make up the elk stack?

"ELK" is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine.

Why use Logstash?

Logstash allows you to collect data from different systems. However, it also does something even more important: it normalises different schema. What does this mean? Simply put, Logstash allows you to put the data it gathers from your various systems into a single common format.


1 Answers

Things definitely seem to be moving quickly at Google's cloud division and documentation does seem to suffer a bit.

Having said that, the document you linked to also details the limitations -

The request and application logs for your app are collected by a Cloud Logging agent and are kept for a maximum of 90 days, up to a maximum size of 1GB. If you want to store your logs for a longer period or store a larger size than 1GB, you can export your logs to Cloud Storage. You can also export your logs to BigQuery and Pub/Sub for further processing.

It should work out of the box for small to medium sized projects. The built in log viewer is also pretty basic.

From your description, it sounds like you may have specific needs, so you should not assume this will be free. You should factor in costs for Cloud Storage for the logs you want to retain and BigQuery depending on your needs to crunch the logs.

like image 195
alpeware Avatar answered Sep 22 '22 21:09

alpeware