I am attempting to build a Python Lambda function that pulls data from multiple Athena databases using the AWS Wrangler Python library.
wr.athena.read_sql_query('across databases sql query', 'one of databases name')
The query is similar to the following:
select * from data_catalog1.database1.table1 as a
left join data_catalog2.database2.table2 as b on a.id = b.id;
This call is not throwing all kinds of permission errors:
I attempted to change the query, but that didn't help either. The only thing that worked was when I queried the data_catalog itself to list the existing databases or tables.
A). Yes it is possible
A. Go to the AWS Glue data catalog, go to a specific Table, and check "Location", which provides information on a file location.
A. Run the query directly in the Athena console to check for errors. This step can often reveal if permissions or configurations are missing.
The other way is to check CloudTrail Logs and enable Verbose logging in AWS Wrangler to check the issues.
Please let me know any other information.