What is the difference between Kaniko and BuildKit/Buildx?

Question

From what I understand:

• They are both tools to build container images
• The build itself runs in a container
• The build can happen on a remote node, for example in a Kubernetes cluster (Kaniko, BuildKit)
• They both offer advanced features such as layer caching

The differences I can gather:

• Security model (Kaniko)
• BuildKit leverages more recent developments such as cache manifest and manifest lists
• BuildKit supports multiple architectures

What I'm not clear is the extent of the overlap between the 2 set of tools and when one should be used instead of the other. For example, both tools seem to cover well the use case of self hosting a remote image build farm on a Kubernetes cluster.

Answer 1

Overlapping features notwithstanding, the primary differences are these:

╔══════════════════════════════╦════════════╦═════════╗
║                              ║ BuildKit   ║ Kaniko  ║
╠══════════════════════════════╬════════════╬═════════╣
║ build with no root or daemon²║ ✔          ║ ✔       ║
║ build multi-architecture³    ║ ✔          ║         ║
║ remote layer caching⁴        ║ ✔          ║ ✔       ║
║ local layer caching⁵         ║ ✔          ║         ║
╚══════════════════════════════╩════════════╩═════════╝

² Both Kaniko and BuildKit can run daemonless and rootless, though Kaniko is, practically speaking and in my humble opinion, easier to build a container from within a non-root container. Kaniko "builds as a root user within a container in an unprivileged environment", but does not require root or a daemon. BuildKit, when exposed via buildx, requires a privileged docker daemon, but BuildKit requires no daemon or root privileges in its standalone form (with some tooling like RootlessKit).

³ Kaniko does not support multi-architecture builds at the time of writing this. https://docs.docker.com/desktop/multi-arch/#build-multi-arch-images-with-buildx

⁴ BuildKit and Kaniko support registry-based caching. BuildKit, however, requires the registry have support for cache manifest lists.

⁵ BuildKit supports multiple --cache-to options, including local filesystem. https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to

Typically the restraints / features of your build environment or platform would dictate which tool is most appropriate, and if you have both as an option, speed may help you decide (though this should be benchmarked thoroughly).