Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove trailing json comma with command line tools

Tags:

json

bash

sed

I want to remove trailing comma from json as,

{
  "key1": "value1",
  "object": {
    "key2": "value2", // <- remove comma
  },
  "key3": "value3", // <- remove comma
}

I came up with,

tr -d '\n' | \
sed -E 's:,(\s*}):\1:g' | \
jq .

and it works but I want to get this fully in sed.

I came up with,

sed -E '/,\s*$/ { N; s:,\s*(\n\s*},?):\1: }'

which works for above input but fails for

{
  "key1": "value1",
  "object": {
    "key2": "value2",
  },
  "key3": "value3",
  "key4": "value4", // <- remove comma
}

as N reads the next line and starts over from the line after next.

// output sed -E '/,\s*$/ { N;l }' using l/look command
{
  "key1": "value1",\n  "object": {$
  "key1": "value1",
  "object": {
    "key2": "value2",\n  },$
    "key2": "value2",
  },
  "key3": "value3",\n  "key4": "value4",$
  "key3": "value3",
  "key4": "value4",
}

Update:

Adding another example for testing:

{
  "key1": "value1",
  "object1": {
    "object2": {
      "key2": "value2"
    },
  },
  "key3": "value3",
}

Update:

This is working for whatever I've thrown at it.

sed -E -n 'H; x; s:,(\s*\n\s*}):\1:; P; ${x; p}' | \
    sed '1 d'

Explanation:

sed -E -n 'H; x; P; ${x; p}'

-n 'H; x' to get every line appended to the next line in pattern space (except for the last line which is simply printed with ${x; p})

and

s:,(\s*\n\s*}):\1:;

to remove the trailing comma in the pattern space.

like image 831
hIpPy Avatar asked Nov 08 '18 03:11

hIpPy


1 Answers

Since the input seems to be some kind of extension of JSON, you could use a command-line tool intended for such extensions. For example:

$ hjson -j < input.txt

or:

$ any-json --input-format=hjson input.txt

Output in both cases

{
  "key1": "value1",
  "object": {
    "key2": "value2"
  },
  "key3": "value3"
}
like image 107
peak Avatar answered Nov 15 '22 01:11

peak