Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do Go testing.B benchmarks prevent unwanted optimizations?

I've recently started learning Go and I'm trying to implement a map that can be used concurrently by multiple groutines. I want to be able to compare my implementation to a simple sync.Mutex-protected map, or to something like this: https://github.com/streamrail/concurrent-map/blob/master/concurrent_map.go

From using Google Caliper, I assume that a naive approach for benchmarking would allow many unwanted optimizations to trash the actual results. Are the benchmarks that use testing.B employing some of the techniques to avoid that (after all both Go and Caliper are Google projects)? If yes, are they known? If not, what's the best way to microbenchmark in Go?

like image 860
Elsinor Avatar asked May 01 '16 12:05

Elsinor


People also ask

What is benchmarking in Go language?

October 26, 2021 7 min read. A benchmark is a type of function that executes a code segment multiple times and compares each output against a standard, assessing the code's overall performance level.


1 Answers

Converting my comment to an answer.

To be completely accurate, any benchmark should be careful to avoid compiler optimisations eliminating the function under test and artificially lowering the run time of the benchmark.

var result int

func BenchmarkFibComplete(b *testing.B) {
        var r int
        for n := 0; n < b.N; n++ {
                // always record the result of Fib to prevent
                // the compiler eliminating the function call.
                r = Fib(10)
        }
        // always store the result to a package level variable
        // so the compiler cannot eliminate the Benchmark itself.
        result = r
}

Source

The following page can also be useful.

Compiler And Runtime Optimizations

Another interesting read is

One other interesting flag is -N, which will disable the optimisation pass in the compiler.

Source1 Source2

I'm not a 100% sure but the following should disable optimisations ? Someone with more experience needs to confirm it.

go test -gcflags=-N -bench=.
like image 113
John S Perayil Avatar answered Sep 28 '22 14:09

John S Perayil