Concurrency errors when using Spring Transaction management in Tomcat

I am currently working on retrofitting Spring-based declarative transactions to a legacy application. The application is deployed on Tomcat and uses JPA/Hibernate to access a PostgreSQL database, and a homegrown web framework (so switching e.g. to Spring Boot is at this time not an option).

My problem is that after changing all DAOs to use an injected EntityManager, everything works when there is only a single user, but with multiple users, I get exceptions that indicate concurrency problems:

Caused by: java.util.ConcurrentModificationException
    at java.base/java.util.HashMap.forEach(HashMap.java:1424)
    at org.hibernate.resource.jdbc.internal.ResourceRegistryStandardImpl.releaseResources(ResourceRegistryStandardImpl.java:328)
    at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.afterTransaction(AbstractLogicalConnectionImplementor.java:60)
    at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.afterTransaction(LogicalConnectionManagedImpl.java:167)
    at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.afterCompletion(LogicalConnectionManagedImpl.java:293)
    at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:95)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:282)
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)

and

Caused by: java.util.ConcurrentModificationException
    at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1013) ~[?:?]
    at java.util.ArrayList$Itr.next(ArrayList.java:967) ~[?:?]
    at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1054) ~[?:?]
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:602) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[?:?]
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:344) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:107) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1407) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]

and

Caused by: org.hibernate.AssertionFailure: possible non thread safe access to session
    at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:215) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[?:?]
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:344) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:107) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1407) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]
    at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1394) ~[hibernate-core-5.6.8.Final.jar:5.6.8.Final]

This is the superclass all DAOs inherit from to get their EntityManager instance:

public abstract class AbstractDao {
    
    protected EntityManager entityManager;

    protected AbstractDao() {
    }
    
    @PersistenceContext(type = PersistenceContextType.EXTENDED)
    @Scope("request") // just an experiment, should not be necessary AFAIK
    public void setEntityManager(EntityManager entityManager) {
        this.entityManager = entityManager;
    }
}

This is the persistence configuration:

@Configuration
@EnableAsync
@EnableTransactionManagement
public class PersistenceConfiguration {
    @Bean
    public EntityManagerFactory getEntityManagerFactory() {
        EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory("my.package");
        
        return entityManagerFactory;
    }

    @Bean
    public PlatformTransactionManager initTransactionManager() {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setEntityManagerFactory(getEntityManagerFactory());
        transactionManager.setJpaDialect(new HibernateJpaDialect());
        
        return transactionManager;
    }
}

Here is the persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://xmlns.jcp.org/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd" version="2.1">
    <persistence-unit name="de.lexcom.agroparts.persistence" transaction-type="RESOURCE_LOCAL">     
        <provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
        <non-jta-data-source>java:comp/env/jdbc/myapp</non-jta-data-source>
        <shared-cache-mode>NONE</shared-cache-mode>
        <properties>
            <property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect"/>
            <property name="hibernate.connection.driver_class" value="org.postgresql.Driver" />
            <property name="show_sql" value="false"/>
            <property name="hibernate.show_sql" value="false" />
            <property name="hibernate.format_sql" value="false" />
            <property name="hibernate.cache.use_second_level_cache" value="false"/>
            <property name="tomee.jpa.factory.lazy" value="true"/>
        </properties>
    </persistence-unit>
</persistence>

And the configuration in web.xml seems to be exactly what is needed to support the request scope, according to the Spring documentation - I have tried configuring either a RequestContextListener or a RequestContextFilter, and can confirm via debugger that they are called, but neither seems to have an effect.

My understanding, according to this article is that the @PersistenceConfig annotation should provide a proxy which forwards calls to a request-scoped entity manager instance that is set up by the RequestContextListener or RequestContextFilter. For some reason, that doesn't seem to be working and instead every request goes to the same entity manager (or at least the same Hibernate session).

Or is the problem the transaction-type="RESOURCE_LOCAL" in the persistence.xml? According to this Stack Overflow answer it sounds like I have to use transaction-type="JTA" to use @PersistenceContext and have multiple EntityManager instances, but that seems to be copied verbatim from the TomEE documentation (a JEE container), and it is contradicted this article which says that "by default, Spring applications use RESOURCE_LOCAL transactions".

What am I missing?

The upper exception indicates that you are using the same Hibernate Session in multiple threads concurrently.

In Tomcat any servlet request is handled in a separate thread, so when you are on Spring it injects a special proxy instead of a simple EntityManager (see SharedEntityManagerCreator). This proxy will automatically either reuse the existing EntityManager or create a new one depending on the current transaction (which is again thread-scoped).

EntityManager itself represent persistence context which is by default bound to a particular transaction. But in your code you use extended-scoped persistence context which can be used across multiple transactions.

@PersistenceContext(type = PersistenceContextType.EXTENDED)
public void setEntityManager(EntityManager entityManager) {
  this.entityManager = entityManager;
}

This seems to be a culprit of your case: with one user you have one thread and no data races, with multiple ones you EntityManager is accessed in racy way resulting in CME.

To fix the issue rewrite your code as:

public abstract class AbstractDao {
    
  @PersistenceContext
  protected EntityManager entityManager;

  protected AbstractDao() {
  }
}

Also as soon as your are using @EnableTransactionManagement I'd suggest to rid persistence.xml in favor of Java config. See the example here.