Need for lock on key when trying to update Multimap value

We have a use case where we are using the Hazelcast Multimap. We’re spawning one thread each for every element within the Collection for every key. For instance, if key1 has, say, 10 values, then we will spawn 10 threads, one for each of the 10 elements in the collection.

Each of these threads tries to update the element it is working with, in a concurrent manner. My question:

Do I need to lock key1 every time a thread has to update an element?

We have an existing system where we lock the Multimap key when we have to update any element with the collection for any key. Due to some reason, when the size of the value collection is 30+, I intermittently run into an issue where the lock on the key is not released by a thread and the subsequent threads keep waiting for a long time (30+ seconds) for the lock. So if it safe to get rid of the locking to begin with, that’d be great.

It depends on what you are trying to do. For example this code is thread safe.

 MultiMap<Integer, Integer> multiMap = hazelcastClientInstance.getMultiMap(MULTI_MAP_NAME);

final int numThreads = 1000;
CountDownLatch latch = new CountDownLatch(numThreads);

for (int index = 0; index < numThreads; index++) {
  final int value = index;
  new Thread(() -> {
    multiMap.put(KEY, value);
    latch.countDown();
  }).start();
}
latch.await();

because the operation on the data structure is performed by a single thread on hazelcast. It does not matter how many client threads post an operation

But this code is not thread safe. Because as stated in the documentation

The results of concurrently mutating the given Collection are undefined.

MultiMap<Integer, Integer> multiMap = hazelcastClientInstance.getMultiMap(MULTI_MAP_NAME);
multiMap.clear();
multiMap.put(KEY, 0);

final int numThreads = 1000;
CountDownLatch latch = new CountDownLatch(numThreads);

for (int index = 0; index < numThreads; index++) {
  new Thread(() -> {
    TreeSet<Integer> integerSet = new TreeSet<>(multiMap.get(KEY));
    // The clients are working on the collection without a lock
    Integer first = integerSet.first();
    LOGGER.info("first is {}", first);
    Set<Integer> collection = Set.of(first + 1);
    try {
      multiMap.putAllAsync(KEY, collection).toCompletableFuture().get();
    } catch (Exception ignored) {
  }
  latch.countDown();
  }).start();
}
latch.await();

So in summary if the operation is posting a key + value it is thread safe. If the operation is posting a key + collection the operation may need to use a lock on the key