Shared Varnish cache across multiple servers

Question

We have 4 Varnish servers behind a load balancer. By default, each Varnish server has its own cache. They do not share a cache. This is a problem because it takes 5-6 page loads of a particular page to get it cached in every server.

To circumvent this problem, I changed the Varnish storage from memory to file. The file is stored on a disk drive which is shared across all the Varnish servers. All Varnish servers are able to start correctly but somehow they are still using their own cache. I did confirm that the cache file is being written into by Varnish.

Any idea how to solve this problem?

Answer 1

I know the question is old, but for future reference:

The file backend stores objects in memory backed by an unlinked file on disk with mmap.

(from https://www.varnish-cache.org/docs/4.0/users-guide/storage-backends.html)

The important word here is unlinked, which means that the file is not available any more outside of the process once it has happened. It can still be used by the creating process which holds a handle on it vie mmap. From the unlink(2) man page:

If the name was the last link to a file but any processes still have the file open the file will remain in existence until the last file descriptor referring to it is closed.

So in a nutshell: no, the same file backend cannot be shared, and besides that, there is currently no known way to share the cache itself between several varnish instances (most existing techniques involve chaining instances to replicate the cache content, but this is replicating, not sharing).