Do 'side inputs' in Cloud Dataflow support reading from BigQuery views?
Tried to point a side-input to a BigQuery view, instead of directly to a BigQuery table. It does not produce an error, but simply returns 0 rows. View works fine inside BigQuery.
For example, given a view referencing a table 'types_test' with just 1 row:
In BigQuery, it works fine:
But using the view as a side-input in Dataflow return 0 rows:
INFO: Reading from BigQuery table <removed>:CPT_7414_PLAYGROUND.view_test
Mar 20, 2015 11:10:08 PM com.google.cloud.dataflow.sdk.io.BigQueryIO evaluateReadHelper
INFO: Number of records read from BigQuery: 0
Do side-inputs support views in BigQuery, or do you need to do something else to use a view for a side-input?
Solution
Found in the FAQ's, that views are not currently supported.
- How can I replicate a cumulative field?
- What are the most recent StackOverflow datasets on BigQuery?
- Linking GA4 to BigQuery not working after 24 hours
- BigQuery - Export CSV certain columns
- How do I add column names to a select that is doing unnest struct in BigQuery
- Google cloud - pub/sub - write to Bigquery with metadata - missing fields
- Calculate Session Length from Google Analytics data in BigQuery
- BigQuery select last modified time using __TABLES__ from all tables in all datasets?
- Loading BigQuery tables from large pandas DataFrames
- How to write a dataframe to BigQuery and overwrite partition instead of the table?
- Python - How to - Big Query asynchronous tasks
- How to retrieve Performance MAX campaigns Google Ads into Google Big Query
- python open() vs gzip.open() and file mode
- Connecting DataGrip to Google Bigquery using GoogleCloud CLI
- Do 'side inputs' in Cloud Dataflow support reading from BigQuery views?
- GCP Workflow deployment failed: token recognition error at: '“' when using variables
- BigQuery giving error of expression not grouped by when order by is included in the sql statement
- BigQuery Omni, write data to aws S3 : this operation cannot write within the same region aws-eu-west-1
- bigquery export data to aws S3 with airflow
- Invalid time zone in BigQuery
- How to get current TIMESTAMP in UTC from BigQuery?
- Implementing iteration in Standard SQL (BigQuery)
- Format a number to have commas (1000000 -> 1,000,000)
- Long form record view in looker studio
- Vertex AI feature store vs BigQuery
- Permissioning Dataflow to read a BigQuery table that is pointing to Drive?
- Allowed Range in case of Date/Timestamp Partitioned table in bigquery
- What does it cost to query files/data in different GCS storage classes from BigQuery (federated/external query)?
- How to efficiently query BigQuery external table (Bigtable) using keys from a smaller table
- Grant access on BigQuery based on table prefix