Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pattern for saving newline-delimited json (aka linejson, jsonlines, .jsonl files) with python

With Python, I'm saving json documents onto separate lines like this:

from bson import json_util # pymongo

with open('test.json', 'ab') as f:
    for document in documents:
       f.write(json_util.dumps(document)+'\n')

and then reading like this:

with open('test.json') as f:
    for line in f:
        document = json_util.loads(line)

The ease and simplicity make me think that there must be a gotcha? Is this all there is to linejson, aka jsonlines?

like image 404
scharfmn Avatar asked Aug 27 '15 06:08

scharfmn


People also ask

What is line delimited JSON format?

ndjson Newline Delimited JSON NDJSON is a convenient format for storing or streaming structured data that may be processed one record at a time. It works well with unix-style text processing tools and shell pipelines.


1 Answers

Yes, that's all there is to it.

like image 106
Cyphase Avatar answered Sep 25 '22 03:09

Cyphase