google-cloud-platformgoogle-cloud-tasks

GCP Cloud Tasks: shorten period for creating a previously created named task


We are developing a GCP Cloud Task based queue process that sends a status email whenever a particular Firestore doc write-trigger fires. The reason we use Cloud Tasks is so a delay can be created (using scheduledTime property 2-min in the future) before the email is sent, and to control dedup (by using a task-name formatted as: [firestore-collection-name]-[doc-id]) since the 'write' trigger on the Firestore doc can be fired several times as the document is being created and then quickly updated by backend cloud functions.

Once the task's delay period has been reached, the cloud-task runs, and the email is sent with updated Firestore document info included. After which the task is deleted from the queue and all is good.

Except:

If the user updates the Firestore doc (say 20 or 30 min later) we want to resend the status email but are unable to create the task using the same task-name. We get the following error:

409 The task cannot be created because a task with this name existed too recently. For more information about task de-duplication see https://cloud.google.com/tasks/docs/reference/rest/v2/projects.locations.queues.tasks/create#body.request_body.FIELDS.task.

This was unexpected as the queue is empty at this point as the last task completed succesfully. The documentation referenced in the error message says:

If the task's queue was created using Cloud Tasks, then another task with the same name can't be created for ~1hour after the original task was deleted or executed.

Question: is there some way in which this restriction can be by-passed by lowering the amount of time, or even removing the restriction all together?


Solution

  • The short answer is No. As you've already pointed, the docs are very clear regarding this behavior and you should wait 1 hour to create a task with same name as one that was previously created. The API or Client Libraries does not allow to decrease this time.

    Having said that, I would suggest that instead of using the same Task ID, use different ones for the task and add an identifier in the body of the request. For example, using Python:

    from google.cloud import tasks_v2
    from google.protobuf import timestamp_pb2
    import datetime
    
    def create_task(project, queue, location, payload=None, in_seconds=None):
        client = tasks_v2.CloudTasksClient()
    
        parent = client.queue_path(project, location, queue)
    
        task = {
                'app_engine_http_request': {
                    'http_method': 'POST',
                    'relative_uri': '/task/'+queue
                }
        }
        if payload is not None:
            converted_payload = payload.encode()
    
            task['app_engine_http_request']['body'] = converted_payload
    
        if in_seconds is not None:
            d = datetime.datetime.utcnow() + datetime.timedelta(seconds=in_seconds)
    
            timestamp = timestamp_pb2.Timestamp()
            timestamp.FromDatetime(d)
    
            task['schedule_time'] = timestamp
    
        response = client.create_task(parent, task)
    
        print('Created task {}'.format(response.name))
        print(response)
    
    #You can change DOCUMENT_ID with USER_ID or something to identify the task
    create_task(PROJECT_ID, QUEUE, REGION, DOCUMENT_ID)