We run a Django application on Heroku, and lately we've been seeing some performance issues. We have New Relic APM add-on installed, and I can see that any time there is a peak in response times, the time is mostly spent in what New Relic calls "Request Queuing" (see attached image). enter image description here
Can someone help me figure out what the problem is here? Why are requests queued? Is this a horizontal scaling issue? It does not seem like the Python app itself is
From my quick read through NewRelic [docs] (https://docs.newrelic.com/docs/apm/apm-ui-pages/features/request-queuing-tracking-front-end-time/) on request queuing.
It seems like your application's workers are busy working on existing requests, that they may take some time before they are ready to serve the next request, hence requests are queued or piled up.
You may want to increase the worker count, or horizontally scale your application to have multiple instances of your backend application and loadbalanced so that more workers can then cater to more requests and the request queue can be shortened.
Ofcourse when you do this, you may also want to look at implementing some sort of Database connection pooling so that you use database connections more efficiently. Thats just another problem you might encounter at scale, which I did in my previous projects.
