javaneo4jtransactions

How to batch transaction in Java Neo4J API using CALL { } IN TRANSACTIONS?


I am developing a Java application using Neo4j version 5.20.0 (implementation "org.neo4j:neo4j:5.20.0") and I want to save a list of Person objects in the database.

Currently, I have a method for creating nodes that uses a single large transaction:

private void saveNodes(List<Person> people) {
    try (Transaction tx = graphDb.beginTx()) {
        for (Person person : people) {
            Node node = tx.createNode(NodeLabel.Person);
            node.setProperty("id", person.getId().toString());
            // 10+ properties
        }
        tx.commit();
    }
}

However, I am saving a large number of nodes, and during the operation, I encounter the following error:

Caused by: org.neo4j.memory.MemoryLimitExceededException: The allocation of an extra 2.0 MiB would use more than the limit 11.1 GiB. Currently using 11.1 GiB. dbms.memory.transaction.total.max threshold reached

I understand that this is related to the data volume and transaction size.

My goal is to break this process into smaller transactions to avoid such issues. I know that in Cypher there is a CALL { ... } IN TRANSACTIONS statement that allows processing data in batches.

Is it possible to implement a similar logic in Java code? If so, how can I do that?


Solution

  • You can commit a batch of node creations per transaction like this:

    private static final int BATCH_SIZE = 10000;
    
    private void saveNodes(List<Person> people) {
        int count = 0;
        Transaction tx = graphDb.beginTx();
        try {
            for (Person person : people) {
                Node node = tx.createNode(NodeLabel.Person);
                node.setProperty("id", person.getId().toString());
                // 10+ properties
    
                if (++count % BATCH_SIZE == 0) {
                    tx.commit();
                    tx.close();
                    tx = graphDb.beginTx();
                }
            }
            tx.commit();
        } finally {
            tx.close();
        }
    }