What's the reason for the 42 layer limit in Docker?

Question

At different places I found the information that a docker image can only consist of up to 42 layers. This seems to be a limitation of the used AUFS file system.

Can anybody tell me why this limit exists or does anybody have some documentation explaining this?

Answer 1

I'm starting to suspect that there isn't any such a hard limit.

Create the following python script, and name it "makeDockerfile.py"

with open("Dockerfile", "w") as file:
    file.write("from alpine:3.8\n")
    for i in range(0, 201):
        file.write("run echo {i}\n".format(i=i))

then run python makeDockerfile.py && docker build --tag manylayer . && docker run -it manylayer /bin/sh You'll see that you are running a working container with > 200 layers.

(note, this was tested with linux containers on linux)

Note that this doesn't mean that this many layers are necessarily SUPPORTED, just that they are possible in some cases.

In fact, I've seen containers fail with far fewer than 42 layers, and removing any arbitrary layer seems to fix it. (see https://github.com/docker/for-win/issues/676#issuecomment-462991673 )

EDIT:

thaJeztah, maintainer of Docker, has this to say about it:

The "42 layer limit" on aufs was on older versions of aufs, but should no longer be the case.

However, the 127 layer limit is still there. This is due to a restriction of Linux not accepting more than X arguments to a syscall that's used.

Although this limit can be raised in modern kernels, it's not the default, so if we'd go beyond that maximum, an Image built on one machine may not be usable on another machine.

( see https://github.com/docker/docker.github.io/issues/8230 )

Answer 2

From the Docker BP documentation:

Minimize the number of layers

You need to find the balance between readability (and thus long-term maintainability) of the Dockerfile and minimizing the number of layers it uses. Be strategic and cautious about the number of layers you use.

They also give advices on how to avoid too many layers:

That way you can delete the files you no longer need after they’ve been extracted and you won’t have to add another layer in your image

[..]

Lastly, to reduce layers and complexity, avoid switching USER back and forth frequently.

---

TL;DR: the benefit of minimizing the number of layers can be likened to the benefit of minimizing the number of small files but rather have fewer bigger ones. A docker pull is also faster (try downloading 2048 files of 1kB or one file of 2MB) . and having fewer layers reduces the complexity of an image, hence the maintainability.

As for the 42 limit. Well... I guess they had to come up with a number and they pick this particular one ;)