I am looking at using Cloud Scheduler to trigger daily execution of a python app on Cloud Run. The app pulls data from a third party API and inserts it into BigQuery. The only issue is, that according to many articles I have read, Cloud Run has a 15 minute execution time limit, which is potentially too short.
Looking at the Cloud Run docs though: https://cloud.google.com/run/quotas#cloud_run_limits it seems to suggest that the maximum execution is 60 minutes.
Which is correct? Can cloud run in fact run processes up to 60 minutes? and am I likely to have any issues using Cloud Schedular to trigger Cloud run if the process takes longer than 30 minutes?
That's a common issue: You can use Cloud Run which can run up to 60 minutes' timeout job, but the serverless product which can trigger it are more restrictive (30 minutes for Cloud Scheduler, 10 minutes for PubSub).
If you don't want to perform retries with Cloud Scheduler, you can simply call your Cloud Run service, and that's all. The Cloud Scheduler will consider the scheduling as failed (timeout) but the process on Cloud Run will continue (event if the Cloud Scheduler client stop listening the answer).
If you want to handle the errors and the retries, or if you want to have "successful" status in your Cloud Scheduler UI, the solution is to use Cloud Workflow. The workflow is really simple: call your API with the params (if applicable). Then here the process:
Finally, you have it, not optimized, but it works.