Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does Logstash has a limit size for each event-message?

Tags:

logstash

I am implementing a monitoring tool on servers of my company service. To do that, I am using logstash. Our applications send their logs via a log4net udp appender to logstash (input udp) and then logstash grok them, and send them to elasticsearch. When I display my logs in kibana, I see that some logs are truncated, the last main part is missing (for big logs). So my question is, does Logstash has a size limit for each message-event received. If yes, is it possible to increase the size. I need all my logs and none of them truncated.

like image 367
user2443476 Avatar asked Dec 19 '22 13:12

user2443476


2 Answers

Update for 2021:

The maximum size is set by the buffer_size parameter in the UDP input. https://www.elastic.co/guide/en/logstash/current/plugins-inputs-udp.html#plugins-inputs-udp-buffer_size

The default buffer size is:

  • 65536 bytes on Logstash >= 5.1
  • 8192 bytes on Logstash >= 2.0
  • 4096 bytes on older versions.

A UDP datagram is limited to 65535 bytes, the length header being 16 bits.

like image 171
user5994461 Avatar answered Jan 30 '23 02:01

user5994461


For the udp case, I think that I have found the solution : -increase the buffer_size parameter in udp.rb file.

I cannot test it now, but I will tell you if it works.

like image 23
user2443476 Avatar answered Jan 30 '23 03:01

user2443476