format shell script logs and write to a file

Question

I need to log all outputs / error logs from a shell script into a file with a standard format.

For example for file script.sh:

echo "I want to be a json message!"
echo "I also want to be a json message!"

executed like below:

./script.sh >> jsonmessages.log

should give me a file content like:

{timestamp: '2020-12-31 00:00:00', message: 'I want to be a json message!'}
{timestamp: '2020-12-31 00:00:00', message: 'I also want to be a json message!'}

Answer 1

The best choice to reliably translate a shell string into a valid JSON string is to use a JSON parser/formatter. The most popular one is jq.

Here is an implementation of a timestamped JSON message logger using jq:

#!/usr/bin/env bash

json_logger() {
  while read -r msg || [ -n "$msg" ]; do
    jq \
      -nc \
      --arg msg "$msg" \
      '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": $msg }'
  done
}


----------

EDIT:

As KamilCuk wrote:

We can do better without slow bash loop - it's just jq --raw-input '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'

So here is an improved json_logger:

json_logger() {
  jq \
    --raw-input \
    --compact-output \
    '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'
}

---

Addendum:

Harshit Kushwaha wrote:

For example, I need to print the method name in json from where the json_logger was called. How can I modify the json_logger and how to use it then in echo?

Here is an implementation with adding the method name if provided as an argument to the json_logger function:

#!/usr/bin/env bash

IFS= read -r -d '' __JQ_LOGGER_SCRIPT <<'JQSCRIPT'
{
  "timestamp": now | strftime("%Y-%m-%d %H:%M:%S"),
  "message": .
} |
  if ($name | length) != 0
  then
    . + { "method": $name }
  else
    .
  end
JQSCRIPT

json_logger() {
  jq \
    --raw-input \
    --compact-output \
    --arg name "$1" \
    "$__JQ_LOGGER_SCRIPT"
}

echo "I want to be a json message!" | json_logger
echo "I also want to be a json message!" | json_logger echo

printf %s $'I am a message with "quoted text" and some special characters: \'\t\7\42\\\'; that can only be properly converted to JSON with a JSON formatter and parser.' | json_logger printf

Produces this JSON outputs:

{"timestamp":"2021-01-29 14:02:46","message":"I want to be a json message!"}
{"timestamp":"2021-01-29 14:02:46","message":"I also want to be a json message!","method":"echo"}
{"timestamp":"2021-01-29 14:02:46","message":"I am a message with \"quoted text\" and some special characters: '\t\u0007\"\\'; that can only be properly converted to JSON with a JSON formatter and parser.","method":"printf"}