[SOLVED] aws cli - download rds postgres logs in a bash script

aws cli - download rds postgres logs in a bash script

I wrote a simple bash script to download my RDS postgres files. But the kicker is that is all works fine tine terminal, but when I try the same thing in the script I get an error:

An error occurred (DBLogFileNotFoundFault) when calling the DownloadDBLogFilePortion operation: DBLog File: "error/postgresql.log.2017-11-05-23", is not found on the DB instance

The command in question is this:

aws rds download-db-log-file-portion --db-instance-identifier foobar --starting-token 0 --output text --log-file error/postgresql.log.2017-11-05-23 >> test.log

It all works fine, but when I put the exact same line in the bash script I get the error message that there are no db log files - which is nonsense, they are there.

This is the bash script:

download_generate_report() {

for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | awk {'print $2'} | grep $2 )
do

echo $filename
echo $1

aws rds download-db-log-file-portion --db-instance-identifier $1 --starting-token 0 --output text --log-file $filename  >> /home/ubuntu/pgbadger_script/postgres_logs/postgres_$1.log.$2
done

}

Tnx, Tom

Solution

I re-wrote your script a little and it seems to work for me. It barked about grep. This uses jq.

for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | jq -r '.DescribeDBLogFiles[] | .LogFileName' )
do

aws rds download-db-log-file-portion --db-instance-identifier $1 --output text --no-paginate --log-file $filename  >> /tmp/postgres_$1.log.$2
done