mysqlgoogle-bigquerygoogle-cloud-sqlfederated-queries

Invalid table-valued function EXTERNAL_QUERY


Something weird has happened with my BigQuery federated queries hitting my Cloud SQL (MySQL) instance. On the 2nd of October all of my scheduled queries started failing. Nothing changed on the database or in BigQuery on the 1st or 2nd, just my scheduled queries started to fail. The schedules have been running for months if not years.

The error I receive in the schedules is:

Query error: Invalid table-valued function EXTERNAL_QUERY Failed to get query schema from MySQL server.

Now this error looks like an authentication/connectivity error, but the password is correct and queries are running when fired against my connection.

My Cloud SQL instance allows connections from public IP:

enter image description here

My queries run when they are being fired from the BigQuery console. This error does appear, but it doesn't prevent the queries from returning successfully:

enter image description here

But my existing scheduled queries are failing:

enter image description here

I created a new scheduled query that's just a carbon copy of the failing one, and it worked. Which is even more bizarre considering all of the details are the same. EDIT: BUT! It only worked once, when it was scheduled it failed again.

I tried updating one of the failing schedules. The query itself had a TRUNCATE and INSERT statement in it, which I removed in favour of handling this in the schedule. But that didn't work either.

I also checked the change log for BiqQuery and nothing was released on the 1st October.

So currently recreating the schedules doesn't work, I can't find any more informative errors, and direct federated queries using the existing connection work, but the schedules continue to fail. Any ideas?

EDIT:

I've checked the Cloud Logs too and the same error message is there without further information.


Solution

  • This is an issue with Query Insights being turned on in Cloud SQL (MySQL).

    Turning off Query Insights resolves the issue.

    A bug has been logged: https://issuetracker.google.com/issues/257873142