Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Check if File exists in S3 using Powershell

I am writing a script that does a Copy-S3Object from S3 using Powershell, however, I need to check the bucket before for a .ready file. The bucket has a folder /test/*.ready. I know how to check my local for a file, but can't figure out how to check the S3:

    Initialize-AWSDefaultConfiguration -AccessKey $AKey -SecretKey $SKey -Region $region

Set-Location $source
$files = Get-ChildItem 'test/*.ready' | Select-Object -Property Name
try {
   if(Test-S3Bucket -BucketName $bucket) {
      foreach($file in $files) {
         if(!(Get-S3Object -BucketName $bucket -Key $file.Name)) { ## verify if exist
            Copy-S3Object -BucketName $bucket -Key $s3File -Region $region -AccessKey $Akey -SecretKey $SKey -LocalFolder $localpath
         } 
      }
   } Else {
      Write-Host "The bucket $bucket does not exist."
   }
} catch {
   Write-Host "Error uploading file $file"
}
like image 333
Mochael_DLite Avatar asked Dec 24 '22 09:12

Mochael_DLite


1 Answers

You can use "Head Object" API to see if the S3 file/Object is created. Here is PowerShell equivalent for HeadObject.

Get-S3ObjectMetadata

The HEAD operation retrieves metadata from an object without returning the object itself. This operation is useful if you're only interested in an object's metadata. To use HEAD, you must have READ access to the object.

Example

try {$metadata = Get-S3ObjectMetadata -BucketName bucket-name -Key someFile.txt; "Found"} catch { "Not Found" }
like image 52
Sudharsan Sivasankaran Avatar answered Jan 05 '23 00:01

Sudharsan Sivasankaran