Are docker artifacts when running a container stored in host fs or memory?

Question

Context

I write rails applications, doing page fragment caching on production (storing the html result of a specific fragment in order to avoid recomputing it).

The usual way to do this is to store the fragment in memcache or redis. The default cache storing option in rails is to use FS caching (storing the fragment as file), because it has the advantage of having no dependencies (you don't have to configure memcache or redis). It is a less popular option, though, because it's slower than memory caching and you have to clean your cache manually, while you can rely on older keys to be automatically purged using memcache or an adequately configured redis.

Question

After using docker for a while, I realize that purging previous cache files is not a problem anymore: on deploy, a new container is ran, automatically dropping all previous cache files. This is probably slower than using memory storing, for sure, but this has the advantage of not requiring any configuration, which is quite cool when bootstrapping quick side projects.

But then I wonder: is writing in a container fs really writing in the fs, or is it writing in the RAM, instead? This is a concern, because this would mean relying on this could very quickly saturate the RAM, especially with many projects on the same server doing so.

Answer 1

The case here is that Docker is doing whatever the process is doing. Docker is just a fancy way to run a process, not a traditional VM. You probably know that, but I'm just reiterating in order to set the foundation for my answer. Docker obeys what your process wants to do. If you specify that you want your process to use a filesystem cache on the disk, and that process writes to the cache, Docker will not store that result in memory. It will write to the cache on disk. You are correct that ephemeral containers have advantages like you mention here, with the file cache being cleaned out when you spin up a new container.

When your process writes to disk, Docker is writing to its special UnionFS (which can use one of a few different storage backends, such as aufs or btrfs, there are several choices). Docker uses this "special" filesystem for two reasons: To avoid duplication of files from the same base image, and to isolate filesystem changes specific to your process from the shared layers of the base image. The specific term for the mechanism is copy-on-write. The point is that when Docker says it's writing to the filesystem, that's what Docker is doing. It isn't writing stuff to memory and acting like the stuff is being written to the filesystem: It's actually writing the stuff to disk. The subject of storage drivers is a deep and complex one, but the documentation for this stuff is well-written and accurate, as is the case for all Docker documentation generally. Rest assured though that there is no magic going into running your process. You can safely assume Docker is doing what it says it's doing.