javaspring-bootcachingguavain-memory

How to build a thread safe in-memory cache per request scope with Spring Boot


I'd like to implement a thread-safe in-memory cache inside a Spring Boot Application. Basically, each time a request comes in, I need to request info from a remote client while saving the response in memory so it could be re-used through my current request in multiple stages of the process. I have been browsing different ways and would like to get an idea which best suits my case

  1. Using a ThreadLocal to hold my context as ThreadLocal keeps the context per request/thread. However, I researched and felt this approach might be outdated (discussed mostly before 2018 as well as may cause GC leak issues (e.g: if an exception happened during the current request while ThreadLocal content didn't get a chance to clear/evict)

  2. Using a Google Guava Cache, this is my current approach and it seems working fine, but I feel like it may be overused per my requirement as my cache is mostly request scoped. In addition, I have to set a large capacity in case my request threads would be more than cache capacity, which may cause cache content to be invalidated incorrectly (e.g: if cache capacity is 10 and I have 15 requests, however, the first 10 served and stored into the cache but didn't complete, the rest of 5 requests may replace some of the cached content causing first few cached contents invalidated)

  3. Define a component class with a concurrent hashmap - unfortunately, this one might not work in multiple thread requests and hard to manage the capacity. (e.g, I have 3 requests A, B and C with same identity key coming at the same time, ideally, I would only expect request A made an internal call, and request B and C will fetch from the cache. However, if my internal call triggered by A would take time while B and C already arrived at the cache location, I saw 2 additional internal calls made by B and C as well)

Please feel free to provide any suggestions if you have experience as well, thank you very much!


Solution

  • I believe you could try to use spring cache with a custom cache manager annotated with scope request_scope and sync=true. This should cover all your requirements.