How to store or cache values to be served in http requests in Golang?

Question

I am writing some Go web services (also implementing the webserver in Go with http.ListenAndServe). I have a map of structs which I would like to keep in memory (with an approximate data size of 100Kb) to be used by different HTTP requests.

How can this be achieved in Go? I am thinking to use global package variables or caching systems (like memcache/groupcache).

Answer 1

In addition to the answers you've already received, consider making use of receiver-curried method values and http.HandlerFunc.

If your data is data that is loaded before the process starts, you could go with something like this:

type Common struct {
    Data map[string]*Data
}

func NewCommon() (*Common, error) {
    // load data
    return c, err
}

func (c *Common) Root(w http.ResponseWriter, r *http.Request) {
    // handler
}

func (c *Common) Page(w http.ResponseWriter, r *http.Request) {
    // handler
}

func main() {
    common, err := NewCommon()
    if err != nil { ... }

    http.HandleFunc("/", common.Root)
    http.HandleFunc("/page", common.Page)

    http.ListenAndServe(...)
}

This works nicely if all of the Common data is read-only. If the Common data is read/write, then you'll want to have something more like:

type Common struct {
    lock sync.RWMutex
    data map[string]Data // Data should probably not have any reference fields
}

func (c *Common) Get(key string) (*Data, bool) {
    c.lock.RLock()
    defer c.lock.RUnlock()
    d, ok := c.data[key]
    return &d, ok
}

func (c *Common) Set(key string, d *Data) {
    c.lock.Lock()
    defer c.lock.Unlock()
    c.data[key] = *d
}

The rest is basically the same, except instead of accessing the data through the receiver's fields directly, you'd access them through the getters and setters. In a webserver where most of the data is being read, you will probably want an RWMutex, so that reads can be executed concurrently with one another. Another advantage of the second approach is that you've encapsulated the data, so you can add in transparent writes to and/or reads from a memcache or a groupcache or something of that nature in the future if your application grows such a need.

One thing that I really like about defining my handlers as methods on an object is that it makes it much easier to unit test them: you can easily define a table driven test that includes the values you want and the output you expect without having to muck around with global variables.