I have created a one java backend service in vertx java. I used httpClient(io.vertx.core.http.HttpClient
) with connection pooling enabled to connect to external services. I am excepting throughput of 50. For each request to my service I need to connect to external service. Average response time for my service is 4 seconds and external service is around 3 seconds.
Now my questions are
maxPoolSize
and maxWaitQueueSize
values for HttpClient?maxPoolSize
and maxWaitQueueSize
values on memory and cpu?maxPoolSize
and maxWaitQueueSize
?setPipelining
option of HttpClient
?First, beware that maxPoolSize
applies to each destination. So if you want different pool sizes, create different HttpClient
for your backends.
Then, unless you work in a constrained environment, I would recommend to leave maxWaitQueueSize
to the default, which is -1
(unbounded). The size of the queue in memory should be relativeley small given the load you're expecting.
To determine the pool size value you can use Little's law. To support a 50 req/sec throughput with an average 3s service time, you need a pool of 150 connections.
The maximum value you can set for maxPoolSize
depends on how your system is configured. In particular, you need to configure the max number of open file descriptors.
For your use case, I believe you should avoid enabling pipeling. First not all HTTP server support it correctly. Second if the service time varies between 0-3 secs, a response might be hold by the backend because previous requests in the pipeline are not handled yet (head-of-line blocking).