I just upgrade our database plan on Heroku for Postgres. On the new plan we have a lot more connections and I'm trying to make sure we're making full use of them at scale.
Say we configured our Puma server with the 40 threads:
puma -t 40:40
...and I set the pool size to 60 (just for a bit of buffer). My understanding is that because I've preallocated 40 Puma threads, each one will reserve a connection, resulting in 40 active connections. However, if I check the active connections there are only 5.
Am I completely misunderstanding how this works?
I am far from an expert in Puma so I just share my own knowledge.
First if you set the number of threads to 40, then your Puma worker will have 40 threads. Though be careful, because of GIL (or GVL) your Puma worker can have only a single thread doing a Ruby task at once. The 39 remaining threads are just sitting idle. UNLESS they are doing I/O (access to database or such ).
Basically the common knowledge is that after 5 threads, you have no more gain from increasing the number of threads. Maybe this can be pushed to 10 if your app is really I/O oriented but I wouldn't go further..
The real concurrency is set by the number of Puma workers (if you boot Puma in clustered mode). If you set the number of Puma workers to 40 then your app can at least handle 40 users at a time.
But 40 workers requires a huge Heroku Dyno, with quite a bit of RAM. Also if you add 5 threads per Puma worker then you need 200 DB connections !
What about the live DB connections
Due to the above, it is very hard to have a single worker with 40 threads to have them all access the DB at the same time. This is probably why your live DB connections are only 5 (unless you have not redeployed your app after the change). I have a small app and also see a varying number of live DB connections across time.
The buffer
Never do a buffer. You are just blocking connections that can't be accessed by your app. The thread pool should equates the max number of threads.
My question: why so many DB connections ?
What was your goal in increasing the DB connections ? More concurrency ? If you have a small app, with a small web dyno, there is no point to have a big database plan behind.
If you want to scale your app. Get a bigger web dyno. Add more Puma workers while sticking to a number of threads to 5.
When the number of workers multiplied by the number of threads exceeds the number of allowed database connections, then it is time to upgrade the database.
Nota Bene : Rails may use a few connections for its internals. So if you have a database with 20 connnections, a Puma config with 3 workers and 5 threads. It is better to upgrade before adding a fourth worker.