My usual response time is <50ms. Here's my tech:
There are two docker containers (~1GB) running on the VM.
However, the request wouldn't reach the server, and would timeout with 404 with the following exception (not reproducible):
sqlalchemy.exc.InterfaceError: (pg8000.exceptions.InterfaceError) network error
All servers are Azure D series VMs and in the same VNET.
I've had this issue previously, which I was able to solve by having only one docker on that VM. However, in this case both images are lightweight.
There was a recent bug fix in pg8000 version 1.29.5 that may fix the problem you're seeing. Looking at the release notes the relevant bit is:
Fixed bug where pg8000 didn’t handle the case when the number of bytes received from a socket was fewer than requested. This was being interpreted as a network error, but in fact we just needed to wait until more bytes were available.
and the commit itself may be helpful.