javagoogle-bigqueryexternal-tablesfederated-table

BigQuery Java client - how to query external ( federated ) table?


I am using the new google-cloud-bigquery and google-cloud-storage api. I want to query an external table, which I created like this:

ExternalTableDefinition etd = ExternalTableDefinition.newBuilder(bucketPath, schema, FormatOptions.csv()).build();
TableId tableID = TableId.of(dataset, targetTableName);
TableInfo tableInfo = TableInfo.newBuilder(tableID, etd).build();

Now I want to query this table, but to have it as a temporary one, using the QueryRequest:

QueryRequest queryRequest = QueryRequest.newBuilder("select * from table limit 10 ").setUseLegacySql(true).setDefaultDataset(dataset).build();
QueryResponse response = client.query(queryRequest);

But it fails due to a table not exists, which makes sense. I am trying to do something similar to this command line:

bq query --project_id=<project ID> --external_table_definition=wikipedia::/tmp/wikipedia 'select name from wikipedia where name contains "Alex";'

but in Java.

To summarize: how do I create and query an external temporary table through Java client for big query?

Reference from documentation: https://cloud.google.com/bigquery/external-data-sources#temporary-tables


Solution

  • For reference, here is the way to do it:

    public static void main(String[] args) 
    {
    
    BigQuery bigquery = 
    BigQueryOptions.getDefaultInstance().getService();
    
    
    ExternalTableDefinition etd = 
    ExternalTableDefinition.newBuilder("gs://", createSchema(), 
    FormatOptions.csv()).build();
    
    TableId tableId = TableId.of("testdataset", "mytablename");
    
    
    bigquery.create(TableInfo.newBuilder(tableId, etd).build());
    
    
    //Now query the table project.testdataset.mytablename
    
    }