How to implement Memory Pooling in Golang

Question

I implemented an HTTP server in Go.

For each request, I need to create hundreds of objects for a particular struct, and I have ~10 structs like that. So after the request is finished as per Go implementation it will be garbage collected.

So for each request this much amount of memory will be allocated and deallocated.

Instead I wanted to implement memory pooling to improve performance from allocation side as well as GC side also

In the beginning of request, I will take from pool and put them back after the request is served

From the pool implementation side

1. How to allocate and deallocate memory of a particular type of struct?
2. How keep track of information this memory got assigned and other is not?

Any other suggestions to improve performance in case of memory allocation and deallocation?

Answer 1

Note beforehand:

Many suggest to use sync.Pool which is a fast, good implementation for temporary objects. But note that sync.Pool does not guarantee that pooled objects are retained. Quoting from its doc:

Any item stored in the Pool may be removed automatically at any time without notification. If the Pool holds the only reference when this happens, the item might be deallocated.

So if you don't want your objects in the Pool to get garbage collected (which depending on your case might result in more allocations), the solution presented below is better, as values in the channel's buffer are not garbage collected. If your objects are really that big that memory pool is justified, the overhead of the pool-channel will be amortized.

Moreover, sync.Pool does not allow you to limit the number of pooled objects, while the presented solution below naturally does.

---

The simplest memory pool "implementation" is a buffered channel.

Let's say you want a memory pool of some big objects. Create a buffered channel holding pointers to values of such expensive objects, and whenever you need one, receive one from the pool (channel). When you're done using it, put it back to the pool (send on the channel). To avoid accidentally losing the objects (e.g. in case of a panic), use defer statement when putting them back.

Let's use this as the type of our big objects:

type BigObject struct {
    Id        int
    Something string
}

Creating a pool is:

pool := make(chan *BigObject, 10)

The size of the pool is simply the size of the channel's buffer.

Filling the pool with pointers of expensive objects (this is optional, see notes at the end):

for i := 0; i < cap(pool); i++ {
    bo := &BigObject{Id: i}
    pool <- bo
}

Using the pool by many goroutines:

wg := sync.WaitGroup{}
for i := 0; i < 100; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        bo := <-pool
        defer func() { pool <- bo }()
        fmt.Println("Using", bo.Id)
        fmt.Println("Releasing", bo.Id)
    }()
}

wg.Wait()

Try it on the Go Playground.

Note that this implementation blocks if all the "pooled" objects are in use. If you don't want this, you may use select to force creating new objects if all are in use:

var bo *BigObject
select {
case bo = <-pool: // Try to get one from the pool
default: // All in use, create a new, temporary:
    bo = &BigObject{Id:-1}
}

And in this case you don't need to put it back into the pool. Or you may choose to try to put all back into the pool if there's room in the pool, without blocking, again with select:

select {
case pool <- bo: // Try to put back into the pool
default: // Pool is full, will be garbage collected
}

Notes:

Filling the pool prior is optional. If you use select to try to get / put back values from / to the pool, the pool may initially be empty.

You have to make sure you're not leaking information between requests, e.g. make sure you don't use fields and values in your shared objects that were set and belong to other requests.