I'm trying to export the partition by day table to the GCP bucket. My main goal is to create daily files in the GCP.
Here is my simple first test for the extraction:
bq extract --destination_format=CSV test.test_table$20210719 gs://test_partition/part_col=20210716/test_*.csv
test_table is a partition table and I do have 7k rows for the date 2021-07-19.
However, when I run this line, I get this
BigQuery error in extract operation: Error processing job 'test-net:bqjob_1234456789: Not found: Table test-
net:temp.test_partitiontime0210719 was not found in location EU
As you can see above the table name is temp.test_partitiontime0210719 there is no 2 at the beginning of it. Whenever $ sign is included in the table name, the first character after it is being removed.
I've tried to apply this GCP documentation. Also, I have tried this comment, but it hasn't worked for me.
How can I extract the partition table to the GCP for a specific date?
It's a linux side effect. If you put $something linux search for an environment variable with the key something.
To skip that, you can use simple quote ' to say to linux: "Hey, do not evaluate this environment variable token", so do this
bq extract --destination_format=CSV 'test.test_table$20210719' gs://test_partition/part_col=20210716/test_*.csv