Redis auto-expiring list items or workflow that would clean up the "expired" items

I want to build a simple analytics app that would use Redis as a DB. The range can vary, but let's say the maximum expiration is 30 days.

In my mind the schema would look like this

{
  "some/path": [1748417529757, 1748417529757],
  "some/other/path": [1748417529757, 1748417529757],
}

On POST I would like to simply append the timestamp to one of the lists. On GET I would be simply getting all the data the DB has and render it.

What would be the most optimal way to achieve this? I read the there's no such thing as list items expiration in Redis. And I'm not sure if checking every list and doing the cleanup on every request based on the current timestamp is optimal.

Solution

Here's the code I ended up with.

import Redis from "ioredis";
export const redis = new Redis("redisUrl");

// This adds a timestamp to the event list. The "member" variable uses an additional "uuid" to be unique. The list allows only unique items and if there were multiple visits at the same time, they will eliminate each other.

export const pushData = (event: string) => {
  const timestamp = Date.now();
  const member = `${timestamp}-${uuid()}`;
  const key = `visits:${event}`;

  return redis.zadd(key, timestamp, member);
}

export type TData = Record<string, number[]>;
export const redis = new Redis("redisUrl");

export const getData = async(): Promise<TData> => {
  const data: TData = {};
  const prefix = `visits:`;
  let cursor = "0";

  do {
    const [nextCursor, keys] = await redis.scan(cursor, "MATCH", `${prefix}*`, "COUNT", 100);
    cursor = nextCursor;

    // Fetch all the keys. Not the best way to do it on big lists.

    for (const key of keys) {
      const event = key.slice(prefix.length);

      // "WITHSCORES" also fetches the scores (timestamps) of the items and that's exactly what I use to count visits.

      const timestamps = await redis.zrange(key, 0, -1, "WITHSCORES");

      // Every other element gets removed to remove keys and keep scores.

      data[event] = timestamps.filter((_, i) => i % 2 !== 0).map(Number)
    }
  } while (cursor !== "0")

  return data;
}

// This goes through all the lists and eliminates items older than the offset.

export const cleanUpOldData = async () => {
  let offset = 30;

  const cutoff = Date.now() - (offset * 24 * 60 * 60 * 1000);
  const pattern = `visits:*`;
  let cursor = "0";

  do {
    const [nextCursor, keys] = await redis.scan(cursor, "MATCH", pattern, "COUNT", 100);
    cursor = nextCursor;

    for (const key of keys) {
      await redis.zremrangebyscore(key, 0, cutoff);
    }
  } while (cursor !== "0");
}