PowerShell foreach file in folder using a lot of memory

Question

I'm fairly new to PowerShell and programming, so I'm sure this code can be optimized even more.

However, my biggest problem with this code is that it puts it's data from the foreach loop in memory. What is the best way to prevent this?

I started up rewriting this script from a script that I found here: PowerShell: delete files older than x days

Edit: Oh, there is about 60 000~ files in each TargetFolder by the way.

## Set Window Title
$title = $Host.UI.RawUI.WindowTitle
$titdef = $title
$Host.UI.RawUI.WindowTitle = "Checking files. Do NOT close " + "($title)"

## Debug (0=Nothing, 1=Extended log, 2=Extended log+Screen)
$Debug = "2"

## Log file
$log = "C:\status\status.txt"
Add-Content $log "nul"

## Clear log before adding new data 
Remove-Item $log

## Folders to check
$TargetFolder1 = “\\server\c$\BiztalkProjects\UDC\output\ume”
$TargetFolder2 = “\\server\c$\BiztalkProjects\UDC\output\upp”

## Required stuff
$Now = Get-Date
$Days = “2”
$TwoDays = $Now.AddDays(-$Days)
$Folder1 = get-childitem $TargetFolder1 -include *
$Folder2 = get-childitem $TargetFolder2 -include *

## Window Message
"Checking for files newer than $Days days in:"
"$TargetFolder1"
"$TargetFolder2"
""
"This may take 5-15 minutes . . . (Will commit about 250MB~ RAM during the period)"

## Reset variable $OK to "0"
$OK = 0

if ($Debug -eq "2") {"Check1"}
foreach ($File in $Folder1) {
    $FileLastWrite = $File.LastWriteTime
    if ($File.LastWriteTime -le $TwoDays) {
        $OK = $OK + "0"
        if ($Debug -eq "2") {
            "OK + 0 (=$OK) $File ($FileLastWrite)"
            }
    } else {
        $OK = $OK + "1"
        if ($Debug -eq "2") {
            "OK + 1 (=$OK) $File ($FileLastWrite)" 
            }
    }
}
if ($Debug -eq "2") {"Check2"}
foreach ($File in $Folder2) {
    $FileLastWrite = $File.LastWriteTime
    if ($File.LastWriteTime -le $TwoDays) {
        $OK = $OK + "0"
        if ($Debug -eq "2") {
            "OK + 0 (=$OK) $File ($FileLastWrite)"
            }
    } else {
        $OK = $OK + "1"
        if ($Debug -eq "2") {
            "OK + 1 (=$OK) $File ($FileLastWrite)" 
            }
    }
}
if ($OK -gt "0") {
    if ($Debug -eq "2") {
        ""
        "Found $OK file(s) newer than $Days days"
        }
    Add-Content $log "OK"
    if ($Debug -ge "1") {
        Add-Content $log "Found $OK file(s) newer than $Days days"
        }
    }
    else {
        if ($Debug -eq "2") {
        ""
        "Found $OK file(s) newer than $Days days"
        }
    Add-Content $log "Error"
    if ($Debug -ge "1") {
        Add-Content $log "Found $OK file(s) newer than $Days days"
        }
    }
$Host.UI.RawUI.WindowTitle = $titdef

This version works. Thanks to all who helped!

## Set Window Title
$title = $Host.UI.RawUI.WindowTitle
$titdef = $title
$Host.UI.RawUI.WindowTitle = "Checking files. Do NOT close " + "($title)"

## Debug (0=Nothing, 1=Extended log, 2=Extended log+Screen)
$Debug = "2"

## Log file
$log = "C:\status\status.txt"
Add-Content $log "nul"

## Clear log before adding new data 
Remove-Item $log

## Folders to check
$TargetFolder1 = “\\server\c$\BiztalkProjects\UDC\output\ume”
$TargetFolder2 = “\\server\c$\BiztalkProjects\UDC\output\upp”

## Required stuff
$Now = Get-Date
$Days = “2”
$TwoDays = $Now.AddDays(-$Days)

## Window Message
"Checking for files newer than $Days days in:"
"$TargetFolder1"
"$TargetFolder2"
""
"This may take 5-15 minutes . . ."

## Reset variable $OK to "0"
$OK = 0

get-childitem $TargetFolder1,$TargetFolder2 -filter *.xml |where-object {
    $FileLastWrite = $_.LastWriteTime
    if ($FileLastWrite -le $TwoDays) {
        $OK = $OK + "0"
    } else {
        $OK = $OK + "1"
    }
}
if ($OK -gt "0") {
    if ($Debug -eq "2") {
        ""
        "Found $OK file(s) newer than $Days days"
        }
    Add-Content $log "OK"
    if ($Debug -ge "1") {
        Add-Content $log "Found $OK file(s) newer than $Days days"
        }
    }
    else {
        if ($Debug -eq "2") {
        ""
        "Found $OK file(s) newer than $Days days"
        }
    Add-Content $log "Error"
    if ($Debug -ge "1") {
        Add-Content $log "Found $OK file(s) newer than $Days days"
        }
    }
$Host.UI.RawUI.WindowTitle = $titdef

Answer 1

Try:

$Folder1 = get-childitem $TargetFolder1 -filter *.xml

Much more performant than using include (on a large set of files). Or even better:

get-childitem $TargetFolder1 -filter *.xml | % {}

---

EDIT

I see now, your problems are due to include! If you need to use wildcards and nor regular expressions to get the child item, use filter instead. Is much more performant because does not use regular expression matching.

Try:

$Folder1 = get-childitem $TargetFolder1 -filter *

even if it's not very clear what you are filtering there (or you want include)...everything? May be you don't need the wildcard at all.

---

In my humble opinion, the best way to prevent foreach loop in memory is piping. However, due to the nature of your script (many nested controls inside the loop) it's not that easy.

Try with:

$Folder1 | % {
  # ...
}

even if I doubt it will improve performances.

---

or:

get-childitem $TargetFolder1 -include * | % {
  # ...
}

Answer 2

Something like

 $TS=(Get-Date).AddDays(-2)
 $COUNT = (get-childitem -recurse path1,path2 |where-object {$_.LastWriteTime -gt $TS}|measure-object).Count
 if ($COUNT > 0) { Echo "There are $COUNT files" }

might be better (I haven't tested it for performance)