using push queues and flexible environment on Google AppEngine I get 403 (Forbidden) error when a task (to be executed on backend service), created with default service, is executed. The task is successfully pushed to queue, confirmed locally, but the execution of the task(s) fails with log:
INFO 2020-12-24 13:42:39,897 module.py:865] default: "POST /tasks/test-handler HTTP/1.1" 403 31
WARNING 2020-12-24 13:42:39,897 taskqueue_stub.py:2158] Task task2 failed to execute. The task has no remaining retries. Failing permanently after 1 retries and 0 seconds
The same happens both locally and on production. However, if a task is created with a cron job then the execution works just fine. I am using dev_appserver.py with Go 1.11 with the following .yaml definitions:
# backend service
service: backend
runtime: go111
instance_class: F2
inbound_services:
- warmup
- default
handlers:
- url: /tasks/.*
login: admin
redirect_http_response_code: 301
# default app service
service: default
runtime: go111
instance_class: F2
inbound_services:
- warmup
handlers:
- url: /api/.*
script: auto
secure: always
redirect_http_response_code: 301
Initial API request comes to an /api endpoint which then succesfully pushes a queue message using:
t := taskqueue.NewPOSTTask(taskURL, url.Values{
"testParam": {strconv.Itoa(testParam)},
})
if _, err := taskqueue.Add(ctx, t, "test-queue"); err != nil {
return ErrPublishingTaskToQueue
}
My queue.yaml definition (in reality I have many more):
total_storage_limit: 120M
queue:
- name: test-queue
rate: 1/s
bucket_size: 100
max_concurrent_requests: 10
retry_parameters:
task_retry_limit: 1
Any ideas why I'd be getting 403 (Forbidden) statuses on task execution if a task is not created via a cron job? The documentation and existing resources on this matter do not help much :/
Managed to make it work. If anyone struggles with getting 403 responses on task execution for push queues on Google AppEngine make sure that you set the right target service. In my example above I was missing target: backend in queue.yaml:
total_storage_limit: 120M
queue:
- name: test-queue
rate: 1/s
bucket_size: 100
max_concurrent_requests: 10
target: backend
retry_parameters:
task_retry_limit: 1
The problem was that the tasks were created with default service which by default means they hit the default service, but should hit backend service. Unfortunately default service had the required endpoint deployed as well, so I got 403 instead of 404.
More details on the target field:
https://cloud.google.com/appengine/docs/standard/python/config/queueref#target