How to mask sensitive information of POST body in nginx log?

Question

For the convenience of analyzing we save the $request_body field in access log. However, there are some sensitive information inside the post body, such as password or credit card number, exposed in the logs. How can we mask these information?

password=1234asdf  ->  password=****

If I write a nginx module to mask the data, should I write a new log module or should I manipulate the request body before the original log module called?
Or should I use nginx-lua to achieve this goal?
Or is there any other methods?

Answer 1

Use 'echo_read_request_body' command to get the HTTP POST data and then filter the password using 'map' and regex

map $request_body $req_body_start {
    "~(?<nopwd>.*)\&password=[^\&]*.+"  $nopwd;
    default        $request_body;
}

map $request_body $req_body_end {
    "~.*\&password=[^\&]*(?<nopwd1>.+)"  $nopwd1;
    default        '';
}

 map $request_body $req_body_pwd {
    "~.*\&password=[^\&]*.+"  '&password=****';
    default        '';
}

Then define log_format and use it on a server/location level:

log_format  logreqbody  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for" "$req_body_start$req_body_pwd$req_body_end"';

Here is more info: https://www.rstcloud.net/blog/30-how-to-hide-sensitive-post-data-in-nginx-log

Answer 2

One Expression Map that can mask the password field regardless of its position in the argument list

    map $request_body $request_body_masked {
            "~(?<pretext>.*)(?<pass0>password)=[^&]+(?<last>.*)$"  '$pretext$pass0=*****$last';
            default  '';
    }

Tested for following patterns

• password=papaya&id=someuser
• password=papaya&
• password=papaya
• &password=papaya
• id=someuser&password=papaya&remember=1
• id=someuser&password=papaya

However, need to benchmark the performance cost of running the regex map over the body of each request.