Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to convert a huge single line json file to a multi line file without opening it?

Tags:

json

bash

ubuntu

I have a large (around 200Mb) single-line json file and I want to convert this to a more readable multi-line json (or txt) file.

I tried to open the file with text editors like sublime text and it takes forever to open. So, I would like to make the conversion without opening the file.

Therefore, I cannot use the interface suggested in this SO question.

I tried to pretty-print the json file as suggested in this answer by doing the following.

cat myjsonfile.json | python -m json.tool > pretty.json

But the terminal prints the following message and I get an empty pretty.json file.

Extra data: line 1 column 34255 - line 1 column 173769197 (char 34254 - 173769196)

I'm thinking of installing visual basic, just to convert the file. But is there a better and efficient way to do the conversion?

like image 516
Achintha Ihalage Avatar asked Aug 30 '19 14:08

Achintha Ihalage


People also ask

What is multiline JSON file?

October 07, 2022. You can read JSON files in single-line or multi-line mode. In single-line mode, a file can be split into many parts and read in parallel. In multi-line mode, a file is loaded as a whole entity and cannot be split. For further information, see JSON Files.

Does JSON support multi-line strings?

Now you can store multi-line strings in JSON.


2 Answers

The simplest method would be using jq to pretty print the json:

jq . myjsonfile.json > pretty.json

But from the python output, I suspect the json file may be ill-formed.

like image 185
dibery Avatar answered Oct 19 '22 02:10

dibery


alternatively, you can use jtc unix utility to pretty-print your one-liner json:

jtc myjsonfile.json

you can use there -t option to control indentation.
If you like to convert myjsonfile.json from one-liner into pretty-printed, then use option -f:

jtc -f myjsonfile.json

btw, to convert it back to one-liner again: jtc -fr myjsonfile.json

PS> Disclosure: I'm the creator of the jtc - shell cli tool for JSON operations

like image 2
Dmitry Avatar answered Oct 19 '22 02:10

Dmitry