profiling golang runtime.systemstack

Question

I have a networking app written in go. the business logic isn't important.
The problem is I'm struggling to utilize the full capabilites of my machine
and when I profile the applicaiton most of the time is spent in runtime stuff. It seems that ALOT of time is spent under runtime.systemstack
I don't understand what it means or how to increase performance.
I add the profiling of the related slow execution methods.
svg file

Answer 1

If you did cpu profiling (go test -cpuprofile), issue 10609 mentioned "runtime: cpu profile is not useful due to systemstack" (fixed here, for Go 1.5)

You can have a more interesting view with the recent Go 1.10 trace: see "Using Go 1.10 new trace features to debug an integration test"

go test -trace trace.out
go tool trace -http=localhost:8080 trace.out

See an example here. You will see clearly the inactivity period for each of your processors.