How to Enable Docker layer caching in Azure DevOps

Question

I am running the below yaml script to build docker images and push into kubernetes cluster but at the same time I wanted to enable docker layer caching in the azure DevOps while building the yaml script.Could you please explain how to enable or how to add the task in azure devops to do this.

Yaml:

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  tag: 'web'
  DockerImageName: 'boiyaa/google-cloud-sdk-nodejs'


steps:
- task: Docker@2
  inputs:
    command: 'build'
    Dockerfile: '**/Dockerfile'
    tags: 'web'
  
- script: |
    echo ${GCLOUD_SERVICE_KEY_STAGING} > ${HOME}/gcp-key.json
               gcloud auth activate-service-account --key-file ${HOME}/gcp-key.json --project ${GCLOUD_PROJECT_ID_STAGING}
               gcloud container clusters get-credentials ${GCLOUD_PROJECT_CLUSTER_ID_STAGING} \
        --zone ${GCLOUD_PROJECT_CLUSTER_ZONE_STAGING} \
        --project ${GCLOUD_PROJECT_ID_STAGING}
  displayName: 'Setup-staging_credentials'


- bash: bash ./deploy/deploy-all.sh staging
  displayName: 'Deploy_script_staging'

Answer 1

Here's how I fixed this. I just pull the latest version of the image from my registry (Azure Container Registry in my case) to the Azure DevOps hosted agent. Then I add --cache-from to the Docker build arguments pointing to this latest tag which it just downloaded to the local machine/cache.

- task: Docker@2
  inputs:
    containerRegistry: '$(ContainerRegistryName)'
    command: 'login'

- script: "docker pull $(ACR_ADDRESS)/$(REPOSITORY):latest"
  displayName: Pull latest for layer caching
  continueOnError: true # for first build, no cache

- task: Docker@2
  displayName: build
  inputs:
    containerRegistry: '$(ContainerRegistryName)'
    repository: '$(REPOSITORY)'
    command: 'build'
    Dockerfile: './dockerfile '
    buildContext: '$(BUILDCONTEXT)'
    arguments: '--cache-from=$(ACR_ADDRESS)/$(REPOSITORY):latest' 
    tags: |
      $(Build.BuildNumber)
      latest

- task: Docker@2
  displayName: "push"
  inputs:
    command: push
    containerRegistry: "$(ContainerRegistryName)"
    repository: $(REPOSITORY) 
    tags: |
      $(Build.BuildNumber)
      latest

Answer 2

Docker layer caching is not supported in Azure DevOps currently. The reason is stated as below:

In the current design of Microsoft-hosted agents, every job is dispatched to a newly provisioned virtual machine. These virtual machines are cleaned up after the job reaches completion, not persisted and thus not reusable for subsequent jobs. The ephemeral nature of virtual machines prevents the reuse of cached Docker layers.

However:

1. Docker layer caching is possible using self-hosted agents. You can try creating your on-premise agents to run your build pipeline.

   You may need to disable the Job's option 'Allow scripts to access the OAuth token'. For $(System.AccessToken) is passed to docker build using a --build-arg ACCESS_TOKEN=$(System.AccessToken), and its value varies for every run, which will invalidate the cache.

2. You can also you use Cache task and docker save/load commands to upload the saved Docker layer to Azure DevOps server and restore it on the future run. Check this thread for more information.

3. Another workaround as described in this blog is to use --cache-from and --target in your Dockerfile.

If the above workarounds are not satisfying, you can submit a feature request to Microsoft Develop Team. Click Suggest a Feature and choose Azure DevOps.

Answer 3

Edit: as pointed out in the comments, this feature is actually available without BuildKit. There's an example here on how to use a Docker image as the cache source during a build.

By adding the variable DOCKER_BUILDKIT: 1 (see this link) to the pipeline job and installing buildx, I managed to achieve layer caching by storing the cache as a separate image. See this link for some basics

Here's an example step in Azure DevOps

- script: |
    image="myreg.azurecr.io/myimage"
    tag=$(Build.SourceBranchName)-$(Build.SourceVersion)
    cache_tag=cache-$(Build.SourceBranchName)

    docker buildx create --use
    docker buildx build \
      -t "${image}:${tag}"
      --cache-from=type=registry,ref=${image}:${cache_tag}\
      --cache-to=type=registry,ref=${image}:${cache_tag},mode=max \
      --push \
      --progress=plain \
      .
  displayName: Build & push image using remote BuildKit layer cache

This of course will require each run to download the image cache, but for images that have long-running installation steps in the Docker build process this is definitely faster (in our case from about 8 minutes to 2).