Why install logstash if I can just send the data through REST to elasticsearch?

Question

I installed elasticsearch and kibana, and I'm following the tutorial. https://www.elastic.co/guide/en/elasticsearch/reference/current/_index_and_query_a_document.html And I'm perfectly inserting and reading data, e.g.:

PUT /customer/external/1?pretty
{
  "name": "John Doe"
}

So, that makes me wonder, what do I need logstash or filebeats for? My plan is to log each web request on a website to elasticsearch for analytics.

Do I need to install logstash? I don't understand what would I need it for. (I don't plan to store it on a file)I will read the request info(e.g. ip address, time, user_id, etc) from a PHP script and simply send it through a HTTP REST REQUEST...as the example above to the elasticsearch server which will save the data anyway. So, I don't see any reason to store the data on the webserver(that is data duplicity), and If I wanted to, why would I need logstash anyway...I can just read a .log file and send it to elasticsearch....like this example: https://www.elastic.co/guide/en/elasticsearch/reference/current/_exploring_your_data.html

Answer 1

No, you do not have to install Logstash, if you plan to collect, normalize and write your application data yourself. As you correctly assumed, Logstash would be a replacement for your PHP script.

Nevertheless, you might still consider to have a look at Logstash. Since it is developed and maintained by same company taking care of Elastic Search, you could benefit from upcoming changes and optimizations.

As you can read from the introduction, Logstash is a tool to read data from multiple sources, normalize it and write the result to multiple destinations. For more details on which sources, filters and oputputs Logstash offers, you should also take a look at the pipeline documentation.