Conditionally locking method of a stateless EJB

A method of a stateless EJB have to do a read-and-insert operation in transaction. First step is a complex check to test if the insert operation is allowed with the given parameters.

public boolean DoCheckThanCreateAndAddItemToGroup(Item newItem, Group group) {
     // STEP 1: checks if newItem can be created and added to group
     // This is a complex operation with database reading and some logic
     // using the current items of the group
     if (!DoesNewItemFitInGroup(newItem, group)) {
         return false;
     }

     // STEP 2: creates (persists) Item and adds it to Group
     em.persist(newItem);
     newItem.setGroup(group);
     return true;
}

It is obvious that the whole method should be locked to prevent concurrent execution in order not to add two or more items that don't fit together in the group. Default container managed JTA transaction handling does not provide this. The method can run parallel with the same group.

Currently, when running parallel, both clients can pass the check before any of them made the insert operation, so both clients insert its new item. But these items are not fit together in the group: after one is added, other should fail the check and mustn't be added.

Even if I can set transaction-isolation to SERIALIZABLE level (but I don't know how), it won't be a perfect solution because when the method is running concurrently with different groups, locking is needless and would slow down the system unnecessarily.

Other solution would be to put the method into a singleton EJB with @Lock(LockType.WRITE). But the problem is the same: for different Groups locking is unnecessary.

Something like this would solve the problem:

public boolean DoCheckThanCreateAndAddItemToGroup(Item newItem, Group group) {
    lock(group) {
        // STEP 1

        // STEP 2
    }
}

How to achieve this behaviour in a suitable way in the EJB environment? (Maybe in distributed environment, as well.)

Or any other idea, how to correctly handle this problem?

Somehow I wanted to lock the method with the group object. The solution is based on Andrei's idea, but I did not have to modify the Group entity. I just had to add these lines at the start of the stateless session bean method:

Group lockGroup = em.find(Group.class, group.getId());
em.lock(lockGroup, LockModeType.PESSIMISTIC_WRITE);

Pessimistic lock on the entity object ensures that if a client already locked the same entity object, the second client waits at the em.lock line until the first client commits the whole transaction.

Thank you, Andrei!