How does type conversion internally work? What is the memory utilization for the same?

Question

How does Go type conversion internally work?

What is the memory utilisation for a type cast? For example:

var str1 string
str1 = "26MB string data"
byt := []byte(str1)
str2 := string(byt)

whenever I type convert any variable, will it consume more memory?

I am concerned about this because when I try to unmarshall, I get "fatal error: runtime: out of memory"

err = json.Unmarshal([]byte(str1), &obj)

str1 value comes from HTTP response, but read using ioutils.ReadAll, hence it contains the complete response.

Answer 1

It's called conversion in Go (not casting), and this is covered in Spec: Conversions:

Specific rules apply to (non-constant) conversions between numeric types or to and from a string type. These conversions may change the representation of x and incur a run-time cost. All other conversions only change the type but not the representation of x.

So generally converting does not make a copy, only changes the type. Converting to / from string usually does, as string values are immutable, and for example if converting a string to []byte would not make a copy, you could change the content of the string by changing elements of the resulting byte slice.

See related question: Does convertion between alias types in Go create copies?

There are some exceptions (compiler optimizations) when converting to / from string does not make a copy, for details see golang: []byte(string) vs []byte(*string).

If you already have your JSON content as a string value which you want to unmarshal, you should not convert it to []byte just for the sake of unmarshaling. Instead use strings.NewReader() to obtain an io.Reader which reads from the passed string value, and pass this reader to json.NewDecoder(), so you can unmarshal without having to make a copy of your big input JSON string.

This is how it could look like:

input := "BIG JSON INPUT"
dec := json.NewDecoder(strings.NewReader(input))

var result YourResultType
if err := dec.Decode(&result); err != nil {
    // Handle error
}

Also note that this solution can further be optimized if the big JSON string is read from an io.Reader, in which case you can completely omit reading it first, just pass that to json.NewDecoder() directly, e.g.:

dec := json.NewDecoder(jsonSource)

var result YourResultType
if err := dec.Decode(&result); err != nil {
    // Handle error
}