In BigQuery, you can use the bq
command line tool to run queries from the command line. However, large queries must have a destination table, and can't directly dump results.
Is there a way to specify a temporary table to be created as the destination table? Attempting to use a non-prefixed table name, which is how temporary tables are normally specified, doesn't work:
bq query --nouse_legacy_sql --session_id=$sid --format=prettyjson --destination_table tempout 'select 1'
BigQuery error in query operation: Invalid value tempout for destination_table: Cannot determine table described by tempout
The official docs have an example using a regular table but not one with a temporary table. It seems temporary tables (like in sessions) cannot be used this way, but it isn't explicit.
You can use a dataset with a default table expiration, so all tables will act as temporary tables:
bq mk --location=US --dataset --default_table_expiration=3600 project:temp_dataset
Another option is update the expiration of the created table:
bq query --nouse_legacy_sql --session_id=$sid --format=prettyjson --destination_table project:dataset.tempout 'select 1'
bq update --expiration 3600 project:dataset.tempout