[SOLVED] All simple (cyclic) paths query returns too much data -> heap overflow

All simple (cyclic) paths query returns too much data -> heap overflow

I am quite new to Neo4j and using the Neo4j desktop version with the Javascript driver. My graph has about 30.000 nodes and 40.000 edges. The final approach would be to get all simple cycles in the graph but I am facing a heap overflow while trying to execute the following cypher query:

let res = await session.run('MATCH p=(n)-[*2..4]-(n) RETURN nodes(p)')

This is of course because the query tries to return all the simple cycles found which is way to much for my heap and unhappily increasing the memory for node.js isn't an option.

So is there any way I could bypass this problem and get all simple cycles from the graph?

Here is the heap overflow error I got:

/Users/paulus/.nvm/versions/node/v15.10.0/bin/node /Users/paulus/routeplanner/RouteFinder.js

<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->


<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

Solution

It's difficult to give you an exact solution without knowing the data model of your graph, but it may help to use skip/limit to get data in smaller bulks.

This get the first 100 patterns:

MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 0 LIMIT 100

And this the next 100:

MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 100 LIMIT 100