nginx logging $request_body as hexadecimal

Question

I am trying to log the request body of requests to my api and nginx is turning all quotes (and some other characters like spaces and tabs) into hexadecimal characters.

Here is my log format

log_format postdata '{"ts": "$time_iso8601", "status": $status, "req": "$uri", "meth": "$request_method", "body": "$request_body"}';

Here is what gets logged

{"ts": "2015-05-20T15:31:11-07:00", "status": 400, "req": "/v2/track", "meth": "POST", "body": {\x22id\x22:\x22user id\x22}}

How can I prevent this so that the resulting log line is

{"ts": "2015-05-20T15:31:11-07:00", "status": 400, "req": "/v2/track", "meth": "POST", "body": {"id":"user id"}}

Answer 1

Sinse 1.13 there is an "escape=none" parameter that turns off data escaping.

http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

log_format  api_request_log escape=none '[$time_local] $request \n$request_body';

Answer 2

You can't stop from escaping it and will have to post process it.

Python2 example:

line = '{\x22id\x22:\x22user id\x22}'
line.decode('unicode_escape')
>> u'{"id":"user id"}'

Python3 example:

line = '{\x22id\x22:\x22user id\x22}'
bytes(line, 'utf-8').decode('unicode_escape')
>> '{"id":"user id"}'

Ruby example (from https://stackoverflow.com/a/18752208/2398354):

require 'yaml'
line = '{\x22id\x22:\x22user id\x22}'
YAML.load(%Q(---\n"#{line}"\n))
=> "{\"id\":\"user id\"}"

Note: This last example is useful if post processing a file with logstash