Menu
I might be asking for too much here and I think it's just not implemented yet but I have ELK stack setup similar (almost identical) to how this tutorial does it: (https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04) So basically, I have filebeat setup on a client computer, and it sends all the log files in a folder to logstash on the server computer.
What I want to do is be able to have gzip files (with text files in them) and text files a directory, and be able to send them all over to logstash with filebeat. So I need filebeat to send a unziped file, or logstash to be able to look in the gzip file and get the text file inside it.
I looked around the web and this doesn't seem to be implemented yet, but most of what I found was with log forwarder and logstash rather then filebeat, so I'm just hoping maybe there is a way with filebeat.
I might just be being overly hopeful here. If you don't know a method to do this then I'd like to know too.
221
The important difference between Logstash and Filebeat is their functionalities, and Filebeat consumes fewer resources. But in general, Logstash consumes a variety of inputs, and the specialized beats do the work of gathering the data with minimum RAM and CPU.
Filebeat doesn't support gz files right now. But there is an ongoing request for that, you can track the progress here. Show activity on this post. Filebeat will not be able to properly consume contents inside GZIP files.
2 and 3) For collecting logs on remote machines filebeat is recommended since it needs less resources than a logstash instance, you would use the logstash output if you want to parse your logs, add or remove fields or make some enrichment on your data, if you don't need to do anything like that you can use the ...
Filebeat helps keep things simple by offering a lightweight way (low memory footprint) to forward and centralize logs and files, making the use of SSH unnecessary when you have a number of servers, virtual machines, and containers that generate logs.
This is currently not possible with filebeat. An issue with this feature request was opened here: https://github.com/elastic/beats/issues/637
A potential temporary workaround can be found here: https://discuss.elastic.co/t/backfilling-old-logs-with-filebeat/38428
74
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
