Mounting docker run in Azure Pipeline job

Question

I am trying to mount the root of my repository to the docker container in a Azure Pipeline job. On my Windows local I can set the source as $(pwd) successfully, but on my Ubuntu hosted agent I get the following error when using $(Build.SourcesDirectory):

docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /var/vsts/28/s.

The path /var/vsts/28/s is correct, so what is the issue?

Yaml definition:

jobs:
- job: Run
  pool:
    name: 'Docker'
  steps:
  - task: Docker@2
    displayName: Login to ACR
    inputs:
      command: login
      containerRegistry: acrServiceConnection
  - task: PowerShell@2
    inputs:
      displayName: Run pulumi go
      targetType: inline
      script: |
        docker run --rm --mount type=bind,source=$(Build.SourcesDirectory),target=/project myimage:latest

EDIT:

When using an MS hosted agent the above works just fine. This leads me to believe it's a permission issue at the checkout location. Any suggestions for this?

I get the same result on our self-hosted and MS agents when running ls -l:

drwxr-xr-x 8 root root 4096 Sep  9 16:09 folder
-rw-r--r-- 1 root root 1347 Sep  9 16:09 file
-rw-r--r-- 1 root root  178 Sep  9 16:09 ..
-rw-r--r-- 1 root root 5457 Sep  9 16:09 ..

Answer 1

It is not specifically mentioned how the Self Hosted agent is setup, and it is possible to run an Ubuntu agent both installed directly into a VM, but also as a Docker container.

If the agent is running as a Docker container the error is likely coming from that when the inner container starts it references a path that exists only in the outer container, but not on the host.

When you run a Docker container within a pipeline and that pipelines executes within an agent also started a a docker container the following happens

|=============================================|
|                    HOST                     |
|   |------------------------------------|    |
|   |       Outer container (Agent)      |    |  
|   |                                    |    |
|   |------------------------------------|    |
|                                             |
|   |------------------------------------|    |
|   |       Inner container (Agent)      |    |  
|   |       (Started from pipeline)      |    |
|   |------------------------------------|    |
|                                             |
|=============================================|

When a Docker container runs inside another Docker container, they both use host's docker deamon and thus all mount paths reference the host, regardless of if a new container is launched from the host or the outer container

Example 1: Mount a path from the host into the outer container

docker run ... -v <path-on-host>:<path-on-outer-container> ...

Example 2: Mount a path from the host into the inner container

docker run ... -v <path-on-host>:<path-on-inner-container> ...

Example 2: Mount a path from the outer container into the inner container Mounting paths from outer container into the inner one is not possible without a workaround, since both containers run on the daemon on the host.

One could either make sure that there is a "shared" space on the Host that mounts into both the outer container and the inner container. (Be careful to always specify a path valid on the host even if the inner container is started from within the outer)

Another option is the one described in the section Mounting volumes using Docker within a Docker container in the Microsoft documentation on Docker agents:

Declare an ENV variable when the outer container is started:

docker run ... --env DIND_USER_HOME=$HOME ...

After this, we can start the inner container from the outer one:

docker run ... -v $DIND_USER_HOME:<path-on-inner-container> ...

Answer 2

As stated in the comments, this is probably a permission issue. In your edit you show us that the owner of the files is root. In windows this shouldn't really matter because NTFS doesn't support permissions the same way they work on Linux/ext4, it cannot store the permissions, so on windows they are most likely just ignored.

As suggested, try prefixing your docker run with a chown like this:

sudo chown -R $USER $(Build.SourcesDirectory) && docker run ....

or make sure your code is checked in into VC with the correct permissions set.