How to get memory size of variable in Go?

Question

I am curious about the memory cost of map and slice, so I wrote a program to compare the sizes. I get the memory size by unsafe.Sizeof(s), but obviously it is wrong, because when I change the size, the output is the same.

func getSlice(size int) []int {
    t := time.Now()
    s := make([]int, size*2)
    for i := 0; i < size; i++ {
        index := i << 1
        s[index] = i
        s[index+1] = i
    }
    fmt.Println("slice time cost: ", time.Since(t))
    return s
}

func getMap(size int) map[int]int {
    t := time.Now()
    m := make(map[int]int, size)
    for i := 0; i < size; i++ {
        m[i] = i
    }
    fmt.Println("map time cost: ", time.Since(t))
    return m
}

func TestMem(t *testing.T) {
    size := 1000
    s := getSlice(size)
    m := getMap(size)
    fmt.Printf("slice size: %d\n", unsafe.Sizeof(s))
    fmt.Printf("map size: %d\n", unsafe.Sizeof(m))
}

Answer 1

unsafe.SizeOf() and reflect.Type.Size() only return the size of the passed value without recursively traversing the data structure and adding sizes of pointed values.

The slice is relatively a simple struct: reflect.SliceHeader, and since we know it references a backing array, we can easily compute its size "manually", e.g.:

s := make([]int32, 1000)

fmt.Println("Size of []int32:", unsafe.Sizeof(s))
fmt.Println("Size of [1000]int32:", unsafe.Sizeof([1000]int32{}))
fmt.Println("Real size of s:", unsafe.Sizeof(s)+unsafe.Sizeof([1000]int32{}))

Output (try it on the Go Playground):

Size of []int32: 12
Size of [1000]int32: 4000
Real size of s: 4012

Maps are a lot more complex data structures, I won't go into details, but check out this question+answer: Golang: computing the memory footprint (or byte length) of a map

Calculating size of any variable or structure (recursively)

If you want "real" numbers, you may take advantage of the testing tool of Go, which can also perform memory benchmarking. Pass the -benchmem argument, and inside the benchmark function allocate only whose memory you want to measure:

func BenchmarkSlice100(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(100) }
}
func BenchmarkSlice1000(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(1000) }
}
func BenchmarkSlice10000(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(10000) }
}
func BenchmarkMap100(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(100) }
}
func BenchmarkMap1000(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(1000) }
}
func BenchmarkMap10000(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(10000) }
}

(Remove the timing and printing calls from getSlice() and getMap() of course.)

Running with

go test -bench . -benchmem

Output is:

BenchmarkSlice100-4    3000000        471 ns/op        1792 B/op      1 allocs/op
BenchmarkSlice1000-4    300000       3944 ns/op       16384 B/op      1 allocs/op
BenchmarkSlice10000-4    50000      39293 ns/op      163840 B/op      1 allocs/op
BenchmarkMap100-4       200000      11651 ns/op        2843 B/op      9 allocs/op
BenchmarkMap1000-4       10000     111040 ns/op       41823 B/op     12 allocs/op
BenchmarkMap10000-4       1000    1152011 ns/op      315450 B/op    135 allocs/op

B/op values tell you how many bytes were allocated per op. allocs/op tells how many (distinct) memory allocations occurred per op.

On my 64-bit architecture (where the size of int is 8 bytes) it tells that the size of a slice having 2000 elements is roughly 16 KB (in line with 2000 * 8 bytes). A map with 1000 int-int pairs required approximately to allocate 42 KB.

Answer 2

This incurs some marshaling overhead but I've found it's the simplest way during runtime to get the size of a value in go. For my needs the marshaling overhead wasn't a big issue so I went this route.

func getRealSizeOf(v interface{}) (int, error) {
    b := new(bytes.Buffer)
    if err := gob.NewEncoder(b).Encode(v); err != nil {
        return 0, err
    }
    return b.Len(), nil
}