Improve the efficiency of my PowerShell script

Question

The below code searches 400+ numbers from a list.txt file to see if it exists within any files within the folder path specified.

The script is very slow and has yet to complete as it did not complete after 25 minutes of running. The folder we are searching is 507 MB (532,369,408 bytes) and it contains 1,119 Files & 480 Folders. Any help to improve the speed of the search and the efficiency is greatly appreciated.

$searchWords = (gc 'C:\temp\list.txt') -split ','
$results = @()
Foreach ($sw in $searchWords)
{
    $files = gci -path 'C:\Users\david.craven\Dropbox\Asset Tagging\_SJC Warehouse_\_Project Completed_\2018\A*' -filter "*$sw*" -recurse

    foreach ($file in $files)
    {
        $object = New-Object System.Object
        $object | Add-Member -Type NoteProperty –Name SearchWord –Value $sw
        $object | Add-Member -Type NoteProperty –Name FoundFile –Value $file.FullName
        $results += $object
    }

}

$results | Export-Csv C:\temp\output.csv -NoTypeInformation

Answer 1

The following should speed up your task substantially:

If the intent is truly to look for the search words in the file names:

$searchWords = (Get-Content 'C:\temp\list.txt') -split ','
$path = 'C:\Users\david.craven\Dropbox\Facebook Asset Tagging\_SJC Warehouse_\_Project Completed_\2018\A*'

Get-ChildItem -File -Path $path -Recurse -PipelineVariable file |
  Select-Object -ExpandProperty Name |
    Select-String -SimpleMatch -Pattern $searchWords |
      Select-Object @{n='SearchWord'; e='Pattern'},
                    @{n='FoundFile'; e={$file.FullName}} |
        Export-Csv C:\temp\output.csv -NoTypeInformation

---

If the intent is to look for the search words in the files' contents:

$searchWords = (Get-Content 'C:\temp\list.txt') -split ','
$path = 'C:\Users\david.craven\Dropbox\Facebook Asset Tagging\_SJC Warehouse_\_Project Completed_\2018\A*'

Get-ChildItem -File -Path $path -Recurse |
  Select-String -List -SimpleMatch -Pattern $searchWords |
    Select-Object @{n='SearchWord'; e='Pattern'},
                  @{n='FoundFile'; e='Path'} |
      Export-Csv C:\temp\output.csv -NoTypeInformation

---

The keys to performance improvement:

• Perform the search with a single command, by passing all search words to Select-String. Note: -List limits matching to 1 match (by any of the given patterns).

• Instead of constructing custom objects in a script block with New-Object and Add-Member, let Select-Object construct the objects for you directly in the pipeline, using calculated properties.

• Instead of building an intermediate array iteratively with += - which behind the scenes recreates the array every time - use a single pipeline to pipe the result objects directly to Export-Csv.