How to do an unpretty print on pretty JSON file in shell >> serial string JSON >> ES _bulk?

Question

In working with Elasticsearch on AWS EC2, I just hit an issue with bulk indexing. The ES _bulk endpoint requires the files to be basically JSON serial strings with \n terminators on each string; and what I have built using various web APIs and file pre/processing is pretty JSON ie., easily human readable.

Is there a simple shell script method to get all the pretty JSON simply concatenated into strings, without loading up some Java libraries or whatever? I can add tokens to the basic file during pre-processing to tag the desired \n breaks if that helps parsing, but if anyone has a tip on the toolset I would be grateful. I have a feeling there are scripts out there and I know there are some libraries, but I have not found any simple command line tools to do the unpretty printing so far.

Answer 1

You can try the great jq tool for parsing JSON in the shell. To de-pretty print with jq, you can use either method below:

cat pretty-printed.json | jq -c . jq -c . pretty-printed.json 

the -c (or --compact-output) tells it to not pretty print (which is the default). The "." tells it to return the JSON content "as is" unmodified other than the reformatting. It gets dumped back to stdout, so you can redirect output or pipe it to something else.

P.S. I was looking to address the same problem and came to this option.

Answer 2

The answer from D_S_toowhite was not a direct answer but it set me thinking in the right way i.e., the problem is to remove all the white space. I found a very simple way to remove all white space using command line tool tr:

tr -d [:space:] inputfile 

The :space: tag removes all white space, tabs, spaces, vertical tabs etc. So a pretty JSON input like this:-

{     "version" : "4.0",     "success" : true,     "result" :     {             "Focus" : 0.000590008,             "Arc" : 12     } } 

becomes this JSON serial string:

{"version":"4.0","success":true,"result":{"Focus":0.000590008,"Arc":12}} 

I still have to solve the \n terminator but I think that is trivial now at least in my special case, just append after closing bracket pair using sed.