timeoutamazon-redshiftonunloadlarge-data

Redshift UNLOAD statement get's canceled after 19min

I'm trying to UNLOAD to s3 a large (6GB) data set using this utility (https://github.com/awslabs/amazon-redshift-utils/blob/master/src/UnloadCopyUtility/redshift-unload-copy.py) from an EC2 instance.

but the UNLOAD statement gets cancelled after 19min and 60 second (I have tried 2 times already)

pg.OperationalError: ERROR:  Query (712244) cancelled on user's request

I have checked the statement_timeout param in my redshift config and it is set to 0. Anyone can tell me what it stops after 20 min?

Solution

  • This timeout is set right in the code of that utility: https://github.com/awslabs/amazon-redshift-utils/blob/94eb6d388c2b19e1609ef3a602c9bffba9211734/src/UnloadCopyUtility/redshift-unload-copy.py#L44

    set_timeout_stmt = "set statement_timeout = 1200000"

    Just change it to whatever you like.