Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Format for writing a JSON log file?

Are there any format standards for writing and parsing JSON log files?

The problem I see is that you can't have a "pure" JSON log file since you need matching brackets and trailing commas are forbidden. So while the following may be written by an application, it can't be parsed by standard JSON parsers:

[{date:'2012-01-01 02:00:01', severity:"ERROR", msg:"Foo failed"}, {date:'2012-01-01 02:04:02', severity:"INFO", msg:"Bar was successful"}, {date:'2012-01-01 02:10:12', severity:"DEBUG", msg:"Baz was notified"}, 

So you must have some conventions on how to structure your log files in a way that a parser can process them. The easiest thing would be "one log message object per line, newlines in string values are escaped". Are there any existing standards and tools?

like image 487
chiborg Avatar asked May 22 '12 10:05

chiborg


People also ask

What is the format of a log file?

The JSON (JavaScript Object Notation) is a highly readable data-interchange format that has established itself as the standard format for structured logging. It is compact and lightweight, and simple to read and write for humans and machines.

What is the correct format of a JSON file?

JSON Syntax RulesData is in name/value pairs. Data is separated by commas. Curly braces hold objects. Square brackets hold arrays.

How do I write to a file in JSON?

First, to write data to a JSON file, we must create a JSON string of the data with JSON. stringify . This returns a JSON string representation of a JavaScript object, which can be written to a file.


1 Answers

You're not going to write a single JSON object per FILE, you're going to write a JSON object per LINE. Each line can then be parsed individually. You don't need to worry about trailing commas and have the whole set of objects enclosed by brackets, etc. See http://blog.nodejs.org/2012/03/28/service-logging-in-json-with-bunyan/ for a more detailed explanation of what this can look like.

Also check out Fluentd http://fluentd.org/ for a neat toolset to work with.

Edit: this format is now called JSONLines or jsonl as pointed out by @Mnebuerquo below - see http://jsonlines.org/

like image 138
HerbCSO Avatar answered Oct 02 '22 22:10

HerbCSO