javaspring-bootcouchbasecouchbase-java-api

Delete unused keys from CB


We have approx 1 million keys in CB and each key has expiry time of 10 days. Basically after every 10 days we run a schedular to re-insert data in CB for these 1 million keys.

Out of these there are approx 0.5 million keys which are not used. There is no way to identify which keys will be used and which will not.

I have checked if there is a way to define 2 TTL for a key.

  1. Delete record after fixed certain time period. (expiry)
  2. Delete record if key not used for certain time period

Is there some way to delete these unused keys.


Solution

  • This is something that is easily implemented in the application tier. Say you wanted all new keys to be deleted after 90 days and delete any keys that are unused for 30 days.

    When you create the document, add a creation timestamp field and a TTL of 30 days. Then when your application reads/updates a document, you can set a new TTL on the document which is calculated as the timestamp + 90 days.

    After 30 days any unused docs will be expired, documents which have been accessed and updated will have a TTL in 60 additional days.

    Thanks, Ian McCloy (Principal Product Manager, Couchbase)