Docker and file locking

Question

I wrote a simple go application and added a flock system to prevent being running twice at the same time:

import "github.com/nightlyone/lockfile"

lock, err := lockfile.New(filepath.Join(os.TempDir(), "pagerduty-read-api.lock"))
if err != nil {
    panic(err)
}

if err = lock.TryLock(); err != nil {
    fmt.Println("Already running.")
    return
}

defer lock.Unlock()

It works well on my host. On docker, I tried to run it with volume sharing of tmp:

docker run --rm -it -v /tmp:/tmp my-go-binary

But it does not work. I suppose it's because the flock system is not ported on volume sharing.

My question: Does Docker have option to make flock working between running instance? If not, what are my other options to have the same behavior?

Thanks.

Answer 1

This morning I wrote a little Python test program that just writes one million consecutive integers to a file, with flock() locking, obtaining and releasing the lock once for each number appended. I started up 5 containers, each running that test program, and each writing to the same file in a docker volume.

With the locking enabled, the numbers were all written without interfering with each other, and there were exactly 5 million integers in the file. They weren't consecutive when written this way, but that's expected and consistent with flock() working.

Without locking, many of the numbers were written in a manner that indicates the numbers were running afoul of the multitasking sans locking. There were only 3,167,546 numbers in the file and there were 13,357 blank lines. That adds up to the 3,180,903 lines in the file - substantially different than the desired 5,000,000.

While a program cannot definitively prove that there will never be problems just by testing many times, to me that's a pretty convincing argument that Linux flock() works across containers.

Also, it just kinda makes sense that flock would work across containers; containers are pretty much just a shared kernel, distinct pid, distinct file (other than volumes) and distinct IP port space.

I ran my test on a Linux Mint 19.1 system with Linux kernel 4.15.0-20-generic, Docker 19.03.0 - build aeac949 and CPython 3.6.8.

Go is a cool language, but why flock() didn't appear to be working across volumes in your Go program I do not know.

HTH.

Answer 2

I suppose that you want to use docker volume or you may need some other docker volume plugins.

According to this article, Docker Volume File permission and locking, docker volumes only provides a way to define a volume to use by multi containers or use by a container after restarting.

In docker volume plugins, flocker may meet your requirements. Flocker is an open-source Container Data Volume Manager for your Dockerized applications.

BTW, if you are using kubernetes, you may need to learn more about persistent volume, persistent volume claim, storage class.