[SOLVED] Does CREATE OR REPLACE in BigQuery contribute towards a previous Partition Table's quota?

Does CREATE OR REPLACE in BigQuery contribute towards a previous Partition Table's quota?

We have a BigQuery query like:

  create or replace table `{project}`.`{dataset}`.`{table}`
  partition by date

  select {...}

If we run this query a few times in a day, we get an error:

Quota exceeded: Your table exceeded quota for Number of partition modifications to a column partitioned table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors

I've previously loaded partitioned tables with bq load --replace and don't remember having similar errors — suggesting the quota resets for the new table.

How does this work? Does create or replace use a cumulative quota for that table name, but bq load --replace resets the quota on each run?

Solution

Is it expected behaviour for ‘CREATE OR REPLACE’ to contribute towards a previous/overwritten Partition Table's ‘maximum number of partition modifications’ quota?

Per-table quotas are bind to the table name. 'CREATE OR REPLACE' does not change the table name so it will not reset the quota.

Why then does a' bq load --replace' not affect the partition table quota?

'bq load --replace' does consume the quota as well. Both 'load' and 'create or replace' can append data to multiple partitions. The number of partitions is counted as the number of partition modifications.