javacachingcaffeine

Caffeine cache time-based eviction with cache writer

I'm using a Caffeine cache with the following configuration:

datesCache = Caffeine.newBuilder()
                .maximumSize(1000L)
                .expireAfterWrite(1, TimeUnit.HOURS)
                .writer(new CacheWriter<String, Map<String, List<ZonedDateTime>>>() {
                    @Override
                    public void write(@NonNull final String key, @NonNull final Map<String, List<ZonedDateTime>> datesList) {
                        CompletableFuture.runAsync(() -> copyToDatabase(key, datesList), Executors.newCachedThreadPool());
                    }

                    @Override
                    public void delete(@NonNull final String key, @Nullable final Map<String, List<ZonedDateTime>> datesList,
                            @NonNull final RemovalCause removalCause) {
                        System.out.println("Cache key " + key + " got evicted from due to " + removalCause);
                    }
                })
                .scheduler(Scheduler.forScheduledExecutorService(Executors.newSingleThreadScheduledExecutor()))
                .removalListener((key, dateList, removalCause) -> {
                    LOG.info("Refreshing cache key {}.", key);
                    restoreKeys(key);
                })
                .build();

I'm using a CacheWriter to copy the records to distributed database upon writes to the cache if the values satisfy certain conditions. Also, upon eviction I'm using a RemovalListener to call a backend service with the evicted key to keep the records up-to-date.

In order to make this work, I also had to initialize the cache upon booting up the service and I use the put method to insert the values in the cache. I retrieve values from the cache using datesCache.get(key, datesList -> callBackendService(key)) just in case the request is for a key that I didn't get upon initialization.

The API that leverages this cache has periods of very heavy use, and it seems that for some reason the records were getting evicted (in every request?) because the code in the RemovalListener and the CacheWriter got executed every few milliseconds, eventually creating over 25,000 threads and making the service error out.

Can anybody tell me if I'm doing something deadly wrong? Or something painful obvious that is wrong? I feel like I'm deeply misunderstanding Caffeine.

The goal is to have the records in the cache refresh every 1 hr and upon refresh, get the new values from the backend API and persist them in a database if they satisfy certain conditions.

Solution

  • The cache entered in an infinite loop because the RemovalListener is async and therefore, upon heavy load, the cache values were being replaced by a request to an expired key before the RemovalListener could actually refresh the cache.

    Therefore the values will:

    1. Be removed from the cache with a REPLACED removal cause
    2. Call the RemovalListener
    3. Refresh again and be replaced, and then
    4. Go back to #1.

    Solution: Evaluate the RemovalCause in the RemovalListener to ignore REPLACED keys. Method wasEvicted() can be used or compare the enum values themselves.