Go(lang) memory usage: RSIZE growing and VSIZE of 139GB?

Question

I am writing my first webserver/webservices program in Go and I realized that the RSIZE (as shown by the command line program "top") grows after repeating the same request to my webservices. Does that means there is a memory leak?

I also noticed that both my app and the go process on "top" have a VSIZE of 139GB (both of them exactly that size). Is it normal?

I am using Go 1.1.2 on OS X 10.8

Many Thanks

Answer 1

Big VSIZE doesn't mean you're really using physical memory; wouldn't worry about that.

RSIZE growing after a single request also isn't worrying. RAM's reclaimed by garbage collection, which costs CPU cycles, so Go and other GC'd languages wait many requests until they need RAM freed up (or at least until a lot of RAM has been allocated) to run a collection. Fewer collections => less CPU time spent.

Leaks in the usual sense are rare because the GC should eventually free memory you aren't holding a reference to. If you have buffers that grow as needed but never shrink, those can have a leak-like effect, and if you accidentally hold a reference to memory that's really dead you can have problems. But unless the process keeps growing forever, I wouldn't assume you're having that problem here.

Here are some memory-management tips for Go; some indirectly apply to other languages, too:

• You often don't need to worry. Collection is often pretty fast and you often have a lot of RAM to play with relative to the size of your data. Before you dive in, be sure there's a problem there to solve. :)
• runtime.ReadMemStats(ms) can tell you how long your program has spent in GC, along with lots of other useful information like how much you're allocating (see runtime module docs at http://golang.org/pkg/runtime/)
• If you're spending too much time in GC and don't know why, memprofile is the next step; a full example involving giving a program an optional -memprofile flag is on the Go blog: http://blog.golang.org/profiling-go-programs
• In general, you reduce GC's by reducing unneeded allocations, especially allocations of big objects (buffers holding whole HTTP responses, say).
  • Sometimes there are natural ways to do that without complicating your program--you can stream output to a Writer instead of buffering up a big response, or reuse an object across several iterations of a loop instead of allocating each time, for instance.
  • Other times, you can recycle large/often-allocated objects instead of making new ones; the standard sync.Pool package helps with that, and there's a nice general description of recycling (from before sync.Pool was standard) on the CloudFlare blog.