So I have been preparing and giving out interviews. I got asked this question in 2 of the interviews and I couldn't come up with a satisfying answer or maybe not what they wanted to hear.
The question is, let's keep aside the various operations techniques like load balancing, multiple instances, database replication and stuff, what changes you can do in your application i.e the REST API to make it capable of handling huge amount of requests?
What I've thought of it till now is that we can make any database or other API calls async to let them run in separate thread in the background so that the processing can move on to other requests. One of my colleagues said to use cache to minimize on database calls.
And maybe increase the size of thread pool but threads are expensive and there's only so many threads you can create. Also, if all the threads in pool are busy, the other requests are blocked until a thread gets available. So, that really doesn't even seem to be classified as a approach to this problem.
Overall, we concluded that in that case there's not much really that we can do other than to make sure that the API should only be performing light-weight operations, if that makes any sense.
I googled for the same but didn't really found anything much other than operations stuff and thread pooling.
I was wondering if the community could provide their input on this. How'd you handle such scenario?
There are many things you can do to have the best performance in spring here are a few.
Reactive
Utilize a reactive non-blocking thread model. This is very beneficial when your application acts as a pass-through, basically if you receive a rest request, then your application needs to make a rest request to a 3rd party application, utilizing a reactive model will allow those threads to be re-used while the request is processing at the external service.
Statelessness
Make your services stateless
Security
Use a JWT rather than an Oauth token that needs to perform a database lookup every time.
Caching
Cache everything that you can. Use local caching rather than distributed caching wherever possible
DB
Use a fast connection pool like Hikari Ensure that your connection pool has the optimal configuration. Optimize queries. Faster response times means more threads available for processing.
Thread management
Correctly configuring the threadpool used for an executor and make a big difference in application performance. However, this configuration must be based on the available resources of you system and the needs of the application.
Microserice Architecture
If you are correctly using a microservice architecture, each service would only receive requests relating to its domain, this could reduce the needs of the application as a whole.
Server Selection
Use an Undertow or Netty embedded server
Optimize JVM
Optimize JVM memory usage for your application or container.