Powershell Copy Files with a Blacklist (Exclude) and a Whitelist (Include)

Question

I'm translating some msbuild scripts to powershell.

In msbuild, I can generate a blacklist and/or whitelist of files I want to (recursively) copy to a destination folder.

As seen below:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="AllTargetsWrapped">

    <PropertyGroup>
        <!-- Always declare some kind of "base directory" and then work off of that in the majority of cases  -->
        <WorkingCheckout>.</WorkingCheckout>
        <WindowsSystem32Directory>c:\windows\System32</WindowsSystem32Directory>
        <ArtifactDestinationFolder>$(WorkingCheckout)\ZZZArtifacts</ArtifactDestinationFolder>
    </PropertyGroup>

    <Target Name="AllTargetsWrapped">

        <CallTarget Targets="CleanArtifactFolder" />
        <CallTarget Targets="CopyFilesToArtifactFolder" />
    </Target>

    <Target Name="CleanArtifactFolder">

        <RemoveDir Directories="$(ArtifactDestinationFolder)" Condition="Exists($(ArtifactDestinationFolder))"/>
        <MakeDir Directories="$(ArtifactDestinationFolder)" Condition="!Exists($(ArtifactDestinationFolder))"/>
        <Message Text="Cleaning done" />
    </Target>

    <Target Name="CopyFilesToArtifactFolder">

        <ItemGroup>
            <MyExcludeFiles Include="$(WindowsSystem32Directory)\**\EventViewer_EventDetails.xsl" />
        </ItemGroup>

        <ItemGroup>
            <MyIncludeFiles Include="$(WindowsSystem32Directory)\**\*.xsl" Exclude="@(MyExcludeFiles)"/>
            <MyIncludeFiles Include="$(WindowsSystem32Directory)\**\*.xslt" Exclude="@(MyExcludeFiles)"/>
            <MyIncludeFiles Include="$(WindowsSystem32Directory)\**\*.png" Exclude="@(MyExcludeFiles)"/>
            <MyIncludeFiles Include="$(WindowsSystem32Directory)\**\*.jpg" Exclude="@(MyExcludeFiles)"/>            
        </ItemGroup>        

        <Copy
                SourceFiles="@(MyIncludeFiles)"
                DestinationFiles="@(MyIncludeFiles->'$(ArtifactDestinationFolder)\%(RecursiveDir)%(Filename)%(Extension)')"
        />

        </Target>

    </Project>

Can I do the same in powershell?

I have tried the below, but it creates a file called "C:\work9\MsBuildExamples\FileCopyRecursive\PowershellResults" (its a file with no extension, not a directory)

$sourceDirectory = 'c:\windows\System32\*'
$destinationDirectory = 'C:\work9\MsBuildExamples\FileCopyRecursive\PowershellResults'
$excludeFiles = @('EventViewer_EventDetails.xsl')
$includeFiles = @('*.xsl','*.xslt','*.png','*.jpg')


Copy-Item $sourceDirectory $destinationDirectory -Recurse -Include $includeFiles -Exclude $excludeFiles
# -Container:$false

APPEND:

I tried this:

$sourceDirectory = 'c:\windows\System32'
$destinationDirectory = 'C:\work9\MsBuildExamples\FileCopyRecursive\PowershellResults'
$excludeFiles = @('EventViewer_EventDetails.xsl')
$includeFiles = @('*.xsl','*.xslt','*.png','*.jpg')


Copy-Item $sourceDirectory $destinationDirectory -Recurse -Include $includeFiles -Exclude $excludeFiles

(No results, not even a file with no extension)

and I tried this:

$sourceDirectory = 'c:\windows\System32'
$destinationDirectory = 'C:\work9\MsBuildExamples\FileCopyRecursive\PowershellResults'
$excludeFiles = @('EventViewer_EventDetails.xsl')
$includeFiles = @('*.xsl','*.xslt','*.png','*.jpg')


Copy-Item $sourceDirectory $destinationDirectory -Recurse -Include $includeFiles -Exclude $excludeFiles -Container:$false

(No results, not even a file with no extension)

Answer 1

Copy-Item -Recurse, as of Windows PowerShell v5.1 / PowerShell Core 6.2.0, has its quirks and limitations; here's what I found:

^{If you have additional information or corrections, please let us know.}

There are two fundamental ways to call Copy-Item -Recurse:

• (a) specifying a directory path as the source - c:\windows\system32

• (b) using a wildcard expression as the source that resolves to multiple items in the source directory - c:\windows\system32\*

There are two fundamental problems:

• The copying behavior varies based on whether the target directory already exists - see below.

• The -Include parameter does not work properly and neither does -Exclude, though problems are much more likely to arise with -Include; see GitHub issue #8459.

DO NOT USE THE SOLUTIONS BELOW IF YOU NEED TO USE -Include - if you do need -Include, use LotPing's helpful solution.

---

Case (a) - a single directory path as the source

If the source is a single directory (or is the only directory among the items that a wildcard pattern resolved to), Copy-Item implicitly also interprets the destination as a directory.

However, if the destination directory already exists, the copied items will be placed in a subdirectory named for the source directory, which in your case means: C:\work9\MsBuildExamples\FileCopyRecursive\PowershellResults\System32

GitHub issue #2934 that - rightfully - complains about this counter-intuitive behavior

There are two basic workarounds:

If acceptable, remove the destination directory first, if it exists - which is obviously to be done with CAUTION (remove -WhatIf once you're confident that the command works as intended):

# Remove a pre-existing destination directory:
if (Test-Path $destinationDirectory) {
  Remove-Item $destinationDirectory -Recurse -WhatIf
}

# Now Copy-Item -Recurse works as intended.
# As stated, -Exclude works as intended, but -Include does NOT.
Copy-Item $sourceDirectory $destinationDirectory -Recurse

Caveat: Remove-Item -Recurse, regrettably, can intermittently act asynchronously and can even fail - for a robust alternative, see this answer.

If you want to retain a preexisting destination dir. - e.g., if you want to add to contents of the destination directory,

• Create the target dir. on demand; that is, create it only if it doesn't already exist.
• Use Copy-Item to copy the contents of the source directory to the target dir.

# Ensure that the target dir. exists.
# -Force is a quiet no-op, if it already exists.
$null = New-Item -Force -ItemType Directory -LiteralPath $destinationDirectory

# Copy the *contents* of the source directory to the target, using
# a wildcard.
# -Force ensures that *hidden items*, if any, are included too.
# As stated, -Exclude works as intended, but -Include does NOT.
Copy-Item -Force $sourceDirectory/* $destinationDirectory -Recurse

---

Case (b) - a wildcard expression as the source

Note:

• If there's exactly 1 directory among the resolved items, the same rules as in case (a) apply.

• Otherwise, the behavior is only problematic if the target item doesn't exist yet. - see below.

• Therefore, the workaround is to ensure beforehand that the destination directory exists:
  New-Item -Force -Path $destinationDirectory -ItemType Directory

If the target item (-Destination argument) doesn't exist yet:

• If there are multiple directories among the resolved items, Copy-Item copies the first directory, and then fails on the second with the following error message:
  Container cannot be copied onto existing leaf item

• If the source is a single file or resolves to files only, Copy-Item implicitly interprets a non-existent destination as a file.
  With multiple files, this means that a single destination file is created, whose content is the content of the file that happened to be copied last - i.e, there is data loss.