google-bigquerygoogle-cloud-dataflow

Is it possible to update schema while doing a load into an existing BigQuery table from DataFlow?

Basically I am trying to do the equivalent of this CLI in DataFlow:

bq load  --source_format=NEWLINE_DELIMITED_JSON --schema_update_option=ALLOW_FIELD_ADDITION --schema=users.schema.json projectid:dataset.users updates/users.json

Is this possible?

Solution

  • Right now BigQueryIO does not support this experimental option. I've filed a JIRA issue to support it.