Parsing multiple JSON objects in Go

Question

Objects like the below can be parsed quite easily using the encoding/json package.

[ 
    {"something":"foo"},
    {"something-else":"bar"}
]

The trouble I am facing is when there are multiple dicts returned from the server like this :

{"something":"foo"}
{"something-else":"bar"}

This can't be parsed using the code below.

correct_format := strings.Replace(string(resp_body), "}{", "},{", -1)
json_output := "[" + correct_format + "]"

I am trying to parse Common Crawl data (see example).

How can I do this?

Answer 1

Assuming your input is really a series of valid JSON documents, use a json.Decoder to decode them:

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "log"
    "strings"
)

var input = `
{"foo": "bar"}
{"foo": "baz"}
`

type Doc struct {
    Foo string
}

func main() {
    dec := json.NewDecoder(strings.NewReader(input))
    for {
        var doc Doc

        err := dec.Decode(&doc)
        if err == io.EOF {
            // all done
            break
        }
        if err != nil {
            log.Fatal(err)
        }

        fmt.Printf("%+v\n", doc)
    }
}

Playground: https://play.golang.org/p/ANx8MoMC0yq

If your input really is what you've shown in the question, that's not JSON and you have to write your own parser.

Answer 2

Seems like each line is its own json object.

You may get away with the following code which will structure this output into correct json:

package main

import (
    "fmt"
    "strings"
)

func main() {
    base := `{"trolo":"lolo"}
{"trolo2":"lolo2"}`

    delimited := strings.Replace(base, "\n", ",", -1)

    final := "[" + delimited + "]"
    fmt.Println(final)
}

You should be able to use encoding/json library on final now.