Do 'side inputs' in Cloud Dataflow support reading from BigQuery views?
Tried to point a side-input to a BigQuery view, instead of directly to a BigQuery table. It does not produce an error, but simply returns 0 rows. View works fine inside BigQuery.
For example, given a view referencing a table 'types_test' with just 1 row:
In BigQuery, it works fine:
But using the view as a side-input in Dataflow return 0 rows:
INFO: Reading from BigQuery table <removed>:CPT_7414_PLAYGROUND.view_test
Mar 20, 2015 11:10:08 PM com.google.cloud.dataflow.sdk.io.BigQueryIO evaluateReadHelper
INFO: Number of records read from BigQuery: 0
Do side-inputs support views in BigQuery, or do you need to do something else to use a view for a side-input?
Solution
Found in the FAQ's, that views are not currently supported.
- Do 'side inputs' in Cloud Dataflow support reading from BigQuery views?
- GCP Workflow deployment failed: token recognition error at: '“' when using variables
- BigQuery giving error of expression not grouped by when order by is included in the sql statement
- bigquery export data to aws S3 with airflow
- Invalid time zone in BigQuery
- How to get current TIMESTAMP in UTC from BigQuery?
- Implementing iteration in Standard SQL (BigQuery)
- Format a number to have commas (1000000 -> 1,000,000)
- Long form record view in looker studio
- Vertex AI feature store vs BigQuery
- Permissioning Dataflow to read a BigQuery table that is pointing to Drive?
- Allowed Range in case of Date/Timestamp Partitioned table in bigquery
- What does it cost to query files/data in different GCS storage classes from BigQuery (federated/external query)?
- How to efficiently query BigQuery external table (Bigtable) using keys from a smaller table
- Grant access on BigQuery based on table prefix
- ImportError: cannot import name pubsub_v1
- When using Apache Beam locally, how to utilize persistant caching for queries in BigQuery?
- SELECT * with GROUP BY col(s)
- Bigquery - Select a column with not grouping them in group by clause
- What's the difference between BigQuery and Bigtable?
- BigQuery REGEXP_EXTRACT end of URL without unsupported lookback
- SSIS ODBC Google BigQuery (Simba) works in local but return ODBC-1 error when I launch pakcage from SQL Agent
- How to download all data in a Google BigQuery dataset?
- Exporting data from BigQuery into a local PostgreSQL/MySql database
- ValueError: install DBtypes to use this function
- How to calculate ages in BigQuery?
- Connecting Simba ODBC driver for big query issue : REST API Timeout was reached
- Adding new columns to existing bigquery table via API
- How to add a Jinja function to .sqlfluff config
- EXTRACT QUARTER FROM DATE IN BIGQUERY