How can I download a file only if it has changed using powershell

Question

I am using this simple function to download a file:

function DownloadFile([string]$url, [string]$file)
{
    $clnt = new-object System.Net.WebClient
    Write-Host "Downloading from $url to $file " 
    $clnt.DownloadFile($url, $file)
}

It works fine but the script I am using that calls it can be called many times and at present that can mean downloading the file(s) many times.

How can i modify the function to only download if the file doesn't exist locally or the server version is newer (e.g. the LastModifiedDate on the server is greater than the LastModifiedDate locally)?

EDIT: This is what I've got so far, seems to work but would like not to have 2 calls to the server.

function DownloadFile([string]$url, [string]$file)
{
    $downloadRequired = $true
    if ((test-path $file)) 
    {
        $localModified = (Get-Item $file).LastWriteTime 
        $webRequest = [System.Net.HttpWebRequest]::Create($url);
        $webRequest.Method = "HEAD";
        $webResponse = $webRequest.GetResponse()
        $remoteLastModified = ($webResponse.LastModified) -as [DateTime] 
        $webResponse.Close()

        if ($remoteLastModified -gt $localModified)
        {
            Write-Host "$file is out of date"
        }
        else
        {
            $downloadRequired = $false
        }

    }

    if ($downloadRequired)
    {
        $clnt = new-object System.Net.WebClient
        Write-Host "Downloading from $url to $file"
        $clnt.DownloadFile($url, $file)
    }
    else
    {
        Write-Host "$file is up to date."
    }
}

Answer 1

I've been beating this up this week, and came up with this

# ----------------------------------------------------------------------------------------------
# download a file
# ----------------------------------------------------------------------------------------------
Function Download-File {
    Param (
        [Parameter(Mandatory=$True)] [System.Uri]$uri,
        [Parameter(Mandatory=$True )] [string]$FilePath
    )

    #Make sure the destination directory exists
    #System.IO.FileInfo works even if the file/dir doesn't exist, which is better then get-item which requires the file to exist
    If (! ( Test-Path ([System.IO.FileInfo]$FilePath).DirectoryName ) ) { [void](New-Item ([System.IO.FileInfo]$FilePath).DirectoryName -force -type directory)}

    #see if this file exists
    if ( -not (Test-Path $FilePath) ) {
        #use simple download
        [void] (New-Object System.Net.WebClient).DownloadFile($uri.ToString(), $FilePath)
    } else {
        try {
            #use HttpWebRequest to download file
            $webRequest = [System.Net.HttpWebRequest]::Create($uri);
            $webRequest.IfModifiedSince = ([System.IO.FileInfo]$FilePath).LastWriteTime
            $webRequest.Method = "GET";
            [System.Net.HttpWebResponse]$webResponse = $webRequest.GetResponse()

            #Read HTTP result from the $webResponse
            $stream = New-Object System.IO.StreamReader($webResponse.GetResponseStream())
            #Save to file
            $stream.ReadToEnd() | Set-Content -Path $FilePath -Force 

        } catch [System.Net.WebException] {
            #Check for a 304
            if ($_.Exception.Response.StatusCode -eq [System.Net.HttpStatusCode]::NotModified) {
                Write-Host "  $FilePath not modified, not downloading..."
            } else {
                #Unexpected error
                $Status = $_.Exception.Response.StatusCode
                $msg = $_.Exception
                Write-Host "  Error dowloading $FilePath, Status code: $Status - $msg"
            }
        }
    }
}

Answer 2

Last modified is in the HTTP response headers.

Try this:

$clnt.OpenRead($Url).Close();
$UrlLastModified = $clnt.ResponseHeaders["Last-Modified"];

If that's newer than the date on your file, your file is old.

The remote server doesn't have to respond with an accurate date or with the file's actual last modified date, but many will.

GetWebResponse() might be a better way to do this (or more correct way). Using OpenRead() and then Close() immediately afterwards bothers my sensibilities, but I may be crazy. I do mostly work on databases.