Difference between hashicorp packer vs linuxkit

Question

I have used hashicorp packer for building baked VM images.

But was wondering linuxkit too do the same stuff I mean building the baked VM images with the only difference of being more container and kernel centeric.

Want to know the exact difference between the working of these two and there use cases.

Also can there be any usecase using both packer and linuxkit.

Answer 1

I have used both fairly extensively (disclosure: I am a volunteer maintainer for LinuxKit). I used packer for quite some time, and switched almost all of the work I did in packer over to LinuxKit (lkt).

In principle both are open-source tools that serve the same purpose: generate an OS image that can be run. Practically, most use it for VM images to run on vbox, AWS, Azure, GCR, etc., but you can generate an image that will run on bare metal, which I have done as well.

Packer, being older, has a more extensive array of provisioners, builders, plugins, etc. It tries to be fairly broad-based and non-opinionated. Build for everywhere, run any install you want.

LinuxKit runs almost everything - onboot processes and continuous services - in a container. Even the init phase - where the OS image will be booted - is configured by copying files from OCI images.

LinuxKit's strong opinions about how to run and build things can in some ways be restrictive, but also liberating.

The most important differences, in my opinion, are the following:

1. lkt builds up from scratch to the bare minimum you need; Packet builds from an existing OS base.
2. lkt's security surface of attack will be smaller, because it starts not with an existing OS, but with, well, nothing.
3. lkt images can be significantly smaller, because you add in only precisely what you need.
4. lkt builds run locally. Packer essentially spins up a VM (vbox, EC2, whatever), runs some base image, modifies it per your instructions, and then saves it as a new image. lkt just manipulates OCI images by downloading and copying files to create a new image.

I can get to the same net result for differences 1-3 with Packer and LinuxKit, albeit lkt is much less work. E.g. I contributed the getty package to LinuxKit to separate and control when/how getty is launched, and in which namespace. The amount of work to separate and control that in a packer image built on a full OS would have been much harder. Same for the tpm package. Etc.

The biggest difference IMO, though, is step 4. Because Packer launches a VM and runs commands in it, it is much slower and much harder to debug. The same packer image that takes me 10+ mins to build can be 30 seconds in lkt. Your mileage may vary, depending on if the OCI images are downloaded or not, and how complex what you are doing is, but it really has been an order of magnitude faster for me.

Similarly, debugging step by step, or finding an error, running, debugging, and rebuilding, is far harder in a process that runs in a remote VM than it is in a local command: lkt build.

As I said, opinions are my own, but those are the reasons that I moved almost all of my build work to lkt, contributed, and agreed to join the excellent group of maintainers when asked by the team.

At the same time, I am deeply appreciative to HashiCorp for their fantastic toolset. Packer served me well; nowadays, LinuxKit serves me better.