Infinite loop while using redis scan command to delete patterns in golang

I am using scan command in golang to get redis keys by a provided pattern. I am using redis cluster, and therefore in order to avoid missing keys I use ForEachMaster. This is the code I use:

func deleteCacheKeys(ctx context.Context, pattern string, count int64, retries int) error {
    redisClient := redis.NewClusterClient(&redis.ClusterOptions{
        Addrs: []string{redisCluster},
    })
    if err := redisClient.Ping(ctx).Err(); err != nil {
        return err
    }

    var cursor uint64
    err = redisClient.ForEachMaster(ctx, func(ctx context.Context, nodeClient *redis.Client) error {
        for {
            keys, cursor := nodeClient.Scan(ctx, cursor, pattern, count).Val()
            if len(keys) > 0 {
                cmd := nodeClient.Del(ctx, keys...)
                if cmd.Err() != nil {
                    return cmd.Err()
                }
            }
            if cursor == 0 {
                break
            }
        }
        return nil
    })

    return err
}

In this function the tricky part is the count that is used in each node client scan command. When I set it to 1000000, everything works fine. But when I use something lower like 100 or even 100000, this code stucks in a infinite loop (the longest I waited was 30 minutes). When using 1000000 as count it usually takes seconds to delete the pattern.

But we were fine using 1 million until our redis dataset became so large that the infinite loop happend with this count as well. I am currently searching for a safe way to delete these pattern without worrying about this count. And I really want to know the reason to why this even happens.

I have tried setting it to -1 but it still stucks. I also tried using Unlink instead of Del command but the result is the same.

Solution

You got tripped by the effects of the keys, cursor := ... statement inside your loop.

The := operator will actually define all variables to its left as new variables, scoped to the body of the loop only.

So your statement:

    var cursor int64 = 0
    for {
        keys, cursor := nodeClient.Scan(ctx, cursor, pattern, count).Val()
        ...

is actually equivalent to:

    var cursor1 int64 = 0
    for {
        keys, cursor2 := nodeClient.Scan(ctx, cursor1, pattern, count).Val()
        ...

Each of your calls to .Scan() will actually be made with CURSOR 0, and cursor2 may never reach 0.

Here is an illustration of a similar issue: https://go.dev/play/p/77VFOW4ldCE

One of the ways to allow assigning to variables in different scopes is:

use = instead of :=,
make an explicit declaration for your keys variable

    var cursor int64 = 0
    for {
        var keys []string
        // in this statement, no new variable is defined,
        // so 'cursor' refers to the variable 3 lines up
        keys, cursor = nodeClient.Scan(ctx, cursor, pattern, count).Val()