powershellamazon-web-servicesamazon-s3aws-powershell

How to download files from S3 to a local folder

I have a requirement to download files from an AWS S3 bucket to a local folder, count the number of files in the local folder, check against S3, and send an email with the number of files.

I tried to download files from S3 but I am getting an error like get-s3object commandnotfoundexception. How do I resolve this issue?

Here is my code:

# Your account access key - must have read access to your S3 Bucket
$accessKey = "YOUR-ACCESS-KEY"
# Your account secret access key
$secretKey = "YOUR-SECRET-KEY"
# The region associated with your bucket e.g. eu-west-1, us-east-1 etc. (see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions)
$region = "eu-west-1"
# The name of your S3 Bucket
$bucket = "my-test-bucket"
# The folder in your bucket to copy, including trailing slash. Leave blank to copy the entire bucket
$keyPrefix = "my-folder/"
# The local file path where files should be copied
$localPath = "C:\s3-downloads\"    

$objects = Get-S3Object -BucketName $bucket -KeyPrefix $keyPrefix -AccessKey $accessKey -SecretKey $secretKey -Region $region

foreach($object in $objects) {
    $localFileName = $object.Key -replace $keyPrefix, ''
    if ($localFileName -ne '') {
        $localFilePath = Join-Path $localPath $localFileName
        Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile $localFilePath -AccessKey $accessKey -SecretKey $secretKey -Region $region
    }
}
Solution

  • If you have installed the AWS PowerShell Module, you haven't correctly loaded it into your current session. We're identifying this as the issue because the error you specified means that the given cmdlet can't be found.

    Verify first that the module is installed, by any of the options below:

    Load module into an existing session: (PowerShell v3 and v4):

    From the documentation:

    In PowerShell 4.0 and later releases, Import-Module also searches the Program Files folder for installed modules, so it is not necessary to provide the full path to the module. You can run the following command to import the AWSPowerShell module. In PowerShell 3.0 and later, running a cmdlet in the module also automatically imports a module into your session.

    To verify correct installation, add the following command to the beginning of your script:

    PS C:\> Import-Module AWSPowerShell

    Load module into an existing session: (PowerShell v2):

    To verify correct installation, add the following command to the beginning of your script:

    PS C:\> Import-Module "C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell\AWSPowerShell.psd1"

    Open a new session with Windows PowerShell for AWS Desktop Shortcut:

    A shortcut is added to your desktop that starts PowerShell with the correct module loaded into the session. If your installation was successful, this shortcut should be present and should also correctly load the AWS PowerShell module without additional effort from you.

    From the documentation:

    The installer creates a Start Menu group called, Amazon Web Services, which contains a shortcut called Windows PowerShell for AWS. For PowerShell 2.0, this shortcut automatically imports the AWSPowerShell module and then runs the Initialize-AWSDefaults cmdlet. For PowerShell 3.0, the AWSPowerShell module is loaded automatically whenever you run an AWS cmdlet. So, for PowerShell 3.0, the shortcut created by the installer only runs the Initialize-AWSDefaults cmdlet. For more information about Initialize-AWSDefaults, see Using AWS Credentials.

    Further Reading: