Docker COPY with folder wildcards

Question

Given a file structure like this:

project root
|-- X.sln
|-- src
|    |-- Foo
|    |    |-- Foo.fsproj
|    |    |-- Foo.fs
|    |-- Bar
|         |-- Bar.fsproj
|         |-- Bar.fs
|-- test
     |-- Baz
          |-- Baz.fsproj

I'd like to first add all .fsproj files to my Docker image, then run a command, then add the rest of the files. I tried the following, but of course it didn't work:

COPY X.sln .
COPY **/*.fsproj .
RUN dotnet restore
COPY . .
RUN dotnet build

The idea is that after the first two COPY steps, the file tree on the image is like this:

working dir
|-- X.sln
|-- src
|    |-- Foo
|    |    |-- Foo.fsproj
|    |-- Bar
|         |-- Bar.fsproj
|-- test
     |-- Baz
          |-- Baz.fsproj

and the rest of the tree is only added in after RUN dotnet restore.

Is there a way to emulate this behavior, preferably without resorting to scripts outside of the dockerfile?

Answer 1

You can use two RUN commands to solve this problem, using the shell commands (find, sed, and xargs).

Follow the steps:

1. Find all fsproj files, with regex extract the filename without extension and with xargs use this data to create directories with mkdir;
2. Based on the previous script change the regex to create from-to syntax and use the mv command to move files to newly created folders.

Example:

    COPY *.sln ./
    COPY */*.fsproj ./
    RUN find *.fsproj | sed -e 's/.fsproj//g' | xargs mkdir
    RUN find *.fsproj | sed -r -e 's/((.+).fsproj)/.\/\1 .\/\2/g' | xargs -I % sh -c 'mv %'

References:

how to use xargs with sed in search pattern

Answer 2

If you use the dotnet command to manage your solution you can use this piece of code:

1. Copy the solution and all project files to the WORKDIR
2. List projects in the solution with dotnet sln list
3. Iterate the list of projects and move the respective *proj files into newly created directories.

COPY *.sln ./
COPY */*/*.*proj ./
RUN dotnet sln list | \
      tail -n +3 | \
      xargs -I {} sh -c \
        'target="{}"; dir="${target%/*}"; file="${target##*/}"; mkdir -p -- "$dir"; mv -- "$file" "$target"'

Answer 3

One pattern that can be used to achieve what you want without resorting to a script outside the Dockerfile is this:

    COPY <project root> .    
    RUN <command to tar/zip the directory to save a copy inside the container> \
         <command the removes all the files you don't want> \
         dotnet restore \
         <command to unpack tar/zip and restore the files> \
         <command to remove the tar/zip> \
         dotnet build

This would keep all of your operations inside the container. I've bundled them all in one RUN command to keep all of that activity into a single layer of the build. You can break them out if you need to.

Here's just one example on linux of how to recursively remove all files except the ones you don't want: https://unix.stackexchange.com/a/15701/327086. My assumption, based on your example, is that this won't be a costly operation for you.