google-cloud-platformgoogle-bigqueryspring-cloud-gcp-bigquery

Fetch Schedule data from a BigQuery Table to another BigQuery Table (Scheduled queries)

I am really new to GCP and I am trying to Query in a GCP BigQuery to fetch all data from one BigQuery table and Insert all into another BigQuery table

I am trying the Following query where Project 1 & Dataset.Table1 is the Project where I am trying to read the data. and Project 2 and Dataset2.Table2 is the Table where I am trying to Insert all the data with the same Naming

SELECT * FROM `Project1.DataSet1.Table1` LIMIT 1000
insert  INTO `Project2.Dataset2.Table2`

But am I receiving a query error message?

Does anyone know how to solve this issue?

Solution

  • There may be a couple of comments...

    1. The syntax might be different => insert into table select and so on - see DML statements in the standard SQL

    2. Such approach of data coping might not be very optimal considering time and cost. It might be better to use bq cp -f ... commands - see BigQuery Copy — How to copy data efficiently between BigQuery environments and bq command-line tool reference - if that is possible in your case.