Are there any format standards for writing and parsing JSON
log files?
The problem I see is that you can't have a "pure" JSON
log file since you need matching brackets and trailing commas are forbidden. So while the following may be written by an application, it can't be parsed by standard JSON parsers:
[{date:'2012-01-01 02:00:01', severity:"ERROR", msg:"Foo failed"}, {date:'2012-01-01 02:04:02', severity:"INFO", msg:"Bar was successful"}, {date:'2012-01-01 02:10:12', severity:"DEBUG", msg:"Baz was notified"},
So you must have some conventions on how to structure your log files in a way that a parser can process them. The easiest thing would be "one log message object per line, newlines in string values are escaped". Are there any existing standards and tools?
The JSON (JavaScript Object Notation) is a highly readable data-interchange format that has established itself as the standard format for structured logging. It is compact and lightweight, and simple to read and write for humans and machines.
JSON Syntax RulesData is in name/value pairs. Data is separated by commas. Curly braces hold objects. Square brackets hold arrays.
First, to write data to a JSON file, we must create a JSON string of the data with JSON. stringify . This returns a JSON string representation of a JavaScript object, which can be written to a file.
You're not going to write a single JSON object per FILE, you're going to write a JSON object per LINE. Each line can then be parsed individually. You don't need to worry about trailing commas and have the whole set of objects enclosed by brackets, etc. See http://blog.nodejs.org/2012/03/28/service-logging-in-json-with-bunyan/ for a more detailed explanation of what this can look like.
Also check out Fluentd http://fluentd.org/ for a neat toolset to work with.
Edit: this format is now called JSONLines or jsonl
as pointed out by @Mnebuerquo below - see http://jsonlines.org/
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With