amazon-web-servicesamazon-redshiftaws-data-pipeline

# of records loaded through AWS Redshift

Is there a way through the AWS console to understand the number of records that got loaded into a redshift table using the AWS data pipeline?

Solution

  • You can get that information from STL_LOAD_COMMITS table.

    Especially look at lines_scanned column:

    Number of lines scanned from the load file. This number may not match the number of rows that are actually loaded. For example, the load may scan but tolerate a number of bad records, based on the MAXERROR option in the COPY command.

    Like

    SELECT * FROM STL_LOAD_COMMITS
ORDER BY curtime DESC
LIMIT 10