Using fluentd in docker to get nginx logs

Question

I have a scenario where nginx is running in one container and fluentd is in another container, i mapped nginx logs to var/logs/nginx directory, but i was unable to retrieve logs to elastic search using fluentd, please help me regarding this:

fluentd.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>
<source>
  @type tail
  path /var/log/nginx/access_in_log
  #pos_file /var/log/td-agent/nginx-access.log.pos
  tag nginx.access
  format nginx
</source>


<match nginx.access>
  @type elasticsearch
  logstash_format true
  host elasticsearchkibana
  port 9200
  index_name nginxindex
  type_name nginxlogtype
</match>

Please let me know what am I missing.

Answer 1

I solved this issue by using the nginx syslog driver (http://nginx.org/en/docs/syslog.html).

In my nginx.conf inside the nginx container I have these settings:

http {
...
access_log syslog:server=<FLUENTD_HOST>:<FLUENTD_PORT>,tag=nginx_access;    
error_log syslog:server=<FLUENTD_HOST>:<FLUENTD_PORT>,tag=nginx_error info;  

In my fluent.conf inside my Fluentd container I have this config:

<source>
    @type syslog
    port 5141
    tag "syslog"
</source>

The logs then look like this: