Taking a JSON string, unmarshaling it into a map[string]interface{}, editing, and marshaling it into a []byte seems more complicated then it should be

Question

I'm doing very basic JSON manipulation to learn some Go, and it works, except one thing seems off, I have to write allot of .(map[string]interface{}) and .([]interface{}) to access entries in the JSON, especially if they are children of children of children, etc.

See here (also on Go Playground: https://play.golang.org/p/Wd-pzHqTsU):

package main

import (
    "fmt"
    "encoding/json"
)

func main() {
    JSON := []byte(`{"key1":"val1","key2":{"c1key1":"c1val1"},"key3":[{"c2key1":{"c3key1":"c3val1"}}]}`)
    fmt.Printf("%s\n", JSON)
    var d map[string]interface{}
    json.Unmarshal(JSON, &d)
    fmt.Println(d["key3"].([]interface{})[0].(map[string]interface{})["c2key1"].(map[string]interface{})["c3key1"])
    d["key3"].([]interface{})[0].(map[string]interface{})["c2key1"].(map[string]interface{})["c3key1"] = "change1"
    fmt.Println(d["key3"].([]interface{})[0].(map[string]interface{})["c2key1"].(map[string]interface{})["c3key1"])
    JSON, _ = json.Marshal(d)
    fmt.Printf("%s\n", JSON)
}

Which returns:

{"key1":"val1","key2":{"c1key1":"c1val1"},"key3":[{"c2key1":{"c3key1":"c3val1"}}]}
c3val1
change1
{"key1":"val1","key2":{"c1key1":"c1val1"},"key3":[{"c2key1":{"c3key1":"change1"}}]}

Now in Python I just access key/values directly instead of defining the type of what I'm accessing every time, that is instead of fmt.Println(d["key3"].([]interface{})[0].(map[string]interface{})["c2key1"].(map[string]interface{})["c3key1"]) you do print d["key3"][0]["c2key1"]["c3key1"]

Python example:

import json

JSON = '{"key3": [{"c2key1": {"c3key1": "c3val1"}}], "key2": {"c1key1": "c1val1"}, "key1": "val1"}'
print JSON
d = json.loads(JSON)
print d["key3"][0]["c2key1"]["c3key1"]
d["key3"][0]["c2key1"]["c3key1"] = "change1"
print d["key3"][0]["c2key1"]["c3key1"]
JSON = json.dumps(d)
print JSON

So am I doing this right in Go? And if so, what's the reason for this design? Or if not, how should I do it?

Answer 1

Foreword: I optimized and improved the below solution, and released it as a library here: github.com/icza/dyno.

---

The cleanest way would be to create predefined types (structures struct) that model your JSON, and unmarshal to a value of that type, and you can simply refer to elements using Selectors (for struct types) and Index expressions (for maps and slices).

However if your input is not of a predefined structure, I suggest you the following 2 helper functions: get() and set(). The first one accesses (returns) an arbitrary element specified by an arbitrary path (list of string map keys and/or int slice indices), the second changes (sets) the value specified by an arbitrary path (implementations of these helper functions are at the end of the answer).

You only have to include these 2 functions once in your project/app.

And now using these helpers, the tasks you want to do becomes this simple (just like the python solution):

fmt.Println(get(d, "key3", 0, "c2key1", "c3key1"))
set("NEWVALUE", d, "key3", 0, "c2key1", "c3key1")
fmt.Println(get(d, "key3", 0, "c2key1", "c3key1"))

Output:

change1
NEWVALUE

Try your modified app on the Go Playground.

Note - Further Simplification:

You can even save the path in a variable and reuse it to simplify the above code further:

path := []interface{}{"key3", 0, "c2key1", "c3key1"}

fmt.Println(get(d, path...))
set("NEWVALUE", d, path...)
fmt.Println(get(d, path...))

---

And the implementations of get() and set() are below. Note: checks whether the path is valid is omitted. This implementation uses Type switches:

func get(m interface{}, path ...interface{}) interface{} {
    for _, p := range path {
        switch idx := p.(type) {
        case string:
            m = m.(map[string]interface{})[idx]
        case int:
            m = m.([]interface{})[idx]
        }
    }
    return m
}

func set(v interface{}, m interface{}, path ...interface{}) {
    for i, p := range path {
        last := i == len(path)-1
        switch idx := p.(type) {
        case string:
            if last {
                m.(map[string]interface{})[idx] = v
            } else {
                m = m.(map[string]interface{})[idx]
            }
        case int:
            if last {
                m.([]interface{})[idx] = v
            } else {
                m = m.([]interface{})[idx]
            }
        }
    }
}

Answer 2

No, this is not the most correct way to handle structured JSON data in Go. Instead, it is better to create a "struct hierarchy" and unmarshal your JSON into structs. E.g.

type Data struct {
    Key1 string
    Key2 struct {
        C1Key1 string
    }
    Key3 []Key3
}

type Key3 struct {
    C2Key1 struct {
        C3Key1 string
    }
}

This approach:

• gives you more control over how your data will be (un)marshaled (via struct tags and json.Unmarshaler and json.Marshaler interfaces)
• gets you rid of type assertions
• instead, gives you more type safety
• has better performance, since struct access is basically free compared to map access.

Playground: https://play.golang.org/p/9XIh8DX1Ms.