How to generate a JSON log from nginx?

Question

I'm trying to generate a JSON log from nginx.

I'm aware of solutions like this one but some of the fields I want to log include user generated input (like HTTP headers) which need to be escaped properly.

I'm aware of the nginx changelog entries from Oct 2011 and May 2008 that say:

*) Change: now the 0x7F-0x1F characters are escaped as \xXX in an    access_log. *) Change: now the 0x00-0x1F, '"' and '\' characters are escaped as \xXX    in an access_log. 

but this still doesn't help since \xXX is invalid in a JSON string.

I've also looked at the HttpSetMiscModule module which has a set_quote_json_str directive, but this just seems to add \x22 around the strings which doesn't help.

Any idea for other solutions to log in JSON format from nginx?

Answer 1

Finally it looks like we have good way to do this with vanilla nginx without any modules. Just define:

log_format json_combined escape=json   '{'     '"time_local":"$time_local",'     '"remote_addr":"$remote_addr",'     '"remote_user":"$remote_user",'     '"request":"$request",'     '"status": "$status",'     '"body_bytes_sent":"$body_bytes_sent",'     '"request_time":"$request_time",'     '"http_referrer":"$http_referer",'     '"http_user_agent":"$http_user_agent"'   '}'; 

Note that escape=json was added in nginx 1.11.8. http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

Answer 2

You can try to use that one https://github.com/jiaz/nginx-http-json-log - addition module for Nginx.