Trying to use docker buildx
with GitLabs shared runners to build a Docker image that can be run on my Raspberry Pi. Job fails saying git is not in PATH but git is installed in image: docker:stable-git
. Any known fixes or a better way to build an ARM64-compatible image without having to build on the Raspberry Pi itself? (RPi becomes unusable while building on it due to CPU usage)
deploy:
image: docker:stable-git
stage: deploy
services:
- docker:dind
before_script:
- export DOCKER_BUILDKIT=1
- export DOCKER_CLI_EXPERIMENTAL=enabled
- docker build --platform=local -o . git://github.com/docker/buildx
- mv buildx ~/.docker/cli-plugins/docker-buildx
script:
- docker buildx build --platform linux/amd64,linux/arm64 -t $CI_REGISTRY_IMAGE .
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker push $CI_REGISTRY_IMAGE
$ export DOCKER_BUILDKIT=1
$ export DOCKER_CLI_EXPERIMENTAL=enabled
$ docker build --platform=local -o . git://github.com/docker/buildx
#1 [internal] load git source git://github.com/docker/buildx
#1 ERROR: failed to init repo at /var/lib/docker/overlay2/xxx/diff: exec: "git": executable file not found in $PATH
------
> [internal] load git source git://github.com/docker/buildx:
------
failed to solve with frontend dockerfile.v0: failed to resolve dockerfile: failed to build LLB: failed to load cache key: failed to init repo at /var/lib/docker/overlay2/xxx/diff: exec: "git": executable file not found in $PATH
ERROR: Job failed: exit code 1
docker buildx build command provides the same user experience as docker build with many new features like creating scoped builder instances, building against multiple nodes concurrently, outputs configuration, inline build caching, and specifying target platform.
GitLab CI in conjunction with GitLab Runner can use Docker Engine to test and build any application. Docker is an open-source project that allows you to use predefined images to run applications in independent "containers" that are run within a single Linux instance.
You can use GitLab CI/CD with Docker to create Docker images. For example, you can create a Docker image of your application, test it, and publish it to a container registry. To run Docker commands in your CI/CD jobs, you must configure GitLab Runner to support docker commands.
It looks like the issue was that the build was failing due to git not being installed in docker:dind
Was able to work around the issue by cloning Docker/BuildX in a separate stage, running docker build
on it, then exporting it to the deploy stage using artifacts.
buildx:
image: docker:19.03-git
stage: buildx
variables:
GIT_STRATEGY: none
artifacts:
paths:
- buildx
expire_in: 1 hour
services:
- docker:19.03-dind
script:
- export DOCKER_BUILDKIT=1
- git clone git://github.com/docker/buildx ./docker-buildx
- docker build --platform=local -o . ./docker-buildx
deploy:
image: docker:19.03
stage: deploy
services:
- name: docker:19.03-dind
command: ["--experimental"]
before_script:
- mkdir -p ~/.docker/cli-plugins
- mv buildx ~/.docker/cli-plugins/docker-buildx
- docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker buildx create --use --name mybuilder
- docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 --push -t $CI_REGISTRY_IMAGE .
I've decided to update my answer with info about using BuildX in CircleCI due to changes made by GitLab to free accounts as well as comments posted by j0nl1 and Yajo
You can download the BuildX binary and use it in your CircleCI job by doing the following
jobs:
build-and-push:
executor: docker/machine
steps:
- checkout
- run:
name: Remove old installation
command: |
sudo systemctl stop docker.service
sudo apt remove docker-ce docker-ce-cli containerd.io
- docker/install-docker:
install-dir: /usr/bin
- run:
name: Restart docker daemon with experimental features
command: |
sudo bash -c 'echo "{\"experimental\":true}" > /etc/docker/daemon.json'
if [[ $EUID == 0 ]]; then export SUDO=""; else export SUDO="sudo"; fi
$SUDO systemctl unmask docker.service
$SUDO systemctl unmask docker.socket
$SUDO systemctl start docker.service
- run:
name: Install buildx
command: |
BUILDX_VERSION="v0.4.1"
curl -sSLo docker-buildx "https://github.com/docker/buildx/releases/download/$BUILDX_VERSION/buildx-$BUILDX_VERSION.linux-amd64"
chmod a+x docker-buildx
mkdir -p ~/.docker/cli-plugins
mv docker-buildx ~/.docker/cli-plugins/docker-buildx
docker version
docker buildx install
- docker/check:
registry: $DOCKER_REGISTRY
- run:
name: Create buildx profile
command: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx create --use --name mybuilder
- docker/build:
registry: $DOCKER_REGISTRY
image: username/repo
tag: latest
extra_build_args: --platform linux/amd64,linux/arm64,linux/arm/v7 --push
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With