javaapache-zookeeperapache-curatordistributed-lock

Curator Hierarchy Locking (overlapping locks)


I was able to successfully get lock on /ads/lock/0-test1 and then couldn't get a lock on /ads/lock

How can I solve this?

InterProcessMutex lock1 = new InterProcessMutex(client, "/ads/lock/0-test1");
if(lock1.acquire(30000, TimeUnit.MILLISECONDS)){
   InterProcessMutex lock2 = new InterProcessMutex(client, "/ads/lock");

   if(lock2.acquire(30000, TimeUnit.MILLISECONDS)) {  //Failing
   ...
   }
}

Update: This is the essence of what's happening in this https://github.com/Microsoft/Cluster-Partition-Rebalancer-For-Kafka/ZookeeperBackedAdoptionLogicImpl.java Locks at line 250(detailed path) and 299(Root path) are sequential. So, when another instance tries the lock a detailed path (250), the lock fails as the root path (299) is locked. The logic is valid but the root lock is never obtained

Update 2: I wrote a small program to check whether overlapping locks work and It does.

public class LockTesting {
    public static final String ROOT_LOCK = "/locks";
    public static final String CHILD_LOCK = ROOT_LOCK+"/child";
    private static CuratorFramework client;

    public static void main(String[] args) throws Exception {

        client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 30));
        client.start();
        InterProcessMutex lock1 = new InterProcessMutex(client, CHILD_LOCK);
        if (lock1.acquire(30000, TimeUnit.MILLISECONDS)) {
            System.out.println("Child Locked");
            InterProcessMutex lock2 = new InterProcessMutex(client, ROOT_LOCK);
            if (lock2.acquire(30000, TimeUnit.MILLISECONDS)) {
                System.out.println("Root Locked");
            }
        }

    }
}

Solution

  • While not documented explicitly (but see technote 7), the mechanisms used by curator to create the lock depend on the child nodes of a particular znode path. The InterProcessMutex is an implementation of the zookeeper lock recipe, whose documentation does include these details. By trying to use a hierarchical construct like this, you are essentially messing with the internal structure of the lock.

    The path to lock should be considered an "object" whose internal znodes are inaccessible and subject to change.

    Response to Update

    The code in question is indeed an example of this inappropriate use.

    Response to Update2

    Yes, it can sometimes work. But it depends on internal details of the InterProcessMutex implementation. You'll find that with some lock names this will work and with others it will not or you'll have undefined behavior.