Why does json.Unmarshal need a pointer to a map, if a map is a reference type?

Question

I was working with json.Unmarshal and came across the following quirk. When running the below code, I get the error json: Unmarshal(non-pointer map[string]string)

func main() {
    m := make(map[string]string)
    data := `{"foo": "bar"}`
    err := json.Unmarshal([]byte(data), m)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(m)
}

Playground

Looking at the documentation for json.Unmarshal, there is seemingly no indication that a pointer is required. The closest I can find is the following line

Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.

The lines regarding the protocol Unmarshal follows for maps are similarly unclear, as it makes no reference to pointers.

To unmarshal a JSON object into a map, Unmarshal first establishes a map to use. If the map is nil, Unmarshal allocates a new map. Otherwise Unmarshal reuses the existing map, keeping existing entries. Unmarshal then stores key-value pairs from the JSON object into the map. The map's key type must either be a string, an integer, or implement encoding.TextUnmarshaler.

Why must I pass a pointer to json.Unmarshal, especially if maps are already reference types? I know that if I pass a map to a function, and add data to the map, the underlying data of the map will be changed (see the following playground example), which means that it shouldn't matter if I pass a pointer to a map. Can someone clear this up?

Answer 1

As stated in the documentation:

Unmarshal uses the inverse of the encodings that Marshal uses, allocating maps, slices, and pointers as necessary, with ...

Unmarshal may allocates the variable(map, slice, etc.). If we pass a map instead of pointer to a map, then the newly allocated map won't be visible to the caller. The following examples (Go Playground) demonstrates this:

package main

import (
    "fmt"
)

func mapFunc(m map[string]interface{}) {
    m = make(map[string]interface{})
    m["abc"] = "123"
}

func mapPtrFunc(mp *map[string]interface{}) {
    m := make(map[string]interface{})
    m["abc"] = "123"

    *mp = m
}

func main() {
    var m1, m2 map[string]interface{}
    mapFunc(m1)
    mapPtrFunc(&m2)

    fmt.Printf("%+v, %+v\n", m1, m2)
}

in which the output is:

map[], map[abc:123]

If the requirement says that a function/method may allocate a variable when necessary and the newly allocated variable need to be visible to the caller, the solution will be: (a) the variable must be in function's return statement or (b) the variable can be assigned to the function/method argument. Since in go everything is pass by value, in case of (b), the argument must be a pointer. The following diagram illustrates what happen in the above example:

1. At first, both map m1 and m2 point to nil.
2. Calling mapFunc will copy the value pointed by m1 to m resulting m will also point to nil map.
3. If in (1) the map already allocated, then in (2) the address of underlying map data structure pointed by m1 (not the address of m1) will be copied to m. In this case both m1 and m point to the same map data structure, thus modifying map items through m1 will also be visible to m.
4. In the mapFunc function, new map is allocated and assigned to m. There is no way to assign it to m1.

In case of pointer:

1. When calling mapPtrFunc, the address of m2 will be copied to mp.
2. In the mapPtrFunc, new map is allocated and assigned to *mp (not mp). Since mp is pointer to m2, assigning the new map to *mp will change the value pointed by m2. Note that the value of mp is unchanged, i.e. the address of m2.

Answer 2

The other key part of the documentation is this:

To unmarshal JSON into a pointer, Unmarshal first handles the case of the JSON being the JSON literal null. In that case, Unmarshal sets the pointer to nil. Otherwise, Unmarshal unmarshals the JSON into the value pointed at by the pointer. If the pointer is nil, Unmarshal allocates a new value for it to point to.

If Unmarshall accepted a map, it would have to leave the map in the same state whether the JSON were null or {}. But by using pointers, there's now a difference between the pointer being set to nil and it pointing to an empty map.

Note that in order for Unmarshall to be able to "set the pointer to nil", you actually need to pass in a pointer to your map pointer:

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

func main() {
    var m *map[string]string
    data := `{}`
    err := json.Unmarshal([]byte(data), &m)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(m)

    data = `null`
    err = json.Unmarshal([]byte(data), &m)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(m)

    data = `{"foo": "bar"}`
    err = json.Unmarshal([]byte(data), &m)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(m)
}

This outputs:

&map[]
<nil>
&map[foo:bar]