igniteapacheignite

Ignite - out of memory error (Java heap exceed)


My ignite node show this error log How to resolve this?

[01:41:17,950][SEVERE][grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%][GridDirectParser] Failed to read message [msg> Apr 29 01:41:17 localhost service.sh[124976]: java.lang.OutOfMemoryError: Java heap space Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArray(DirectByteBufferStreamImplV> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readByteArray(DirectByteBufferStreamI> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readByteArray(DirectMessageReader.java:189) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.CacheObjectAdapter.readFrom(CacheObjectAdapter.java:145) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateRequest.readFrom(> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:271) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerif> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3734) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNio> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2273) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) Apr 29 01:41:17 localhost service.sh[124976]: at java.base/java.lang.Thread.run(Thread.java:833) [01:41:18,951][SEVERE][grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%][TcpCommunicationSpi] Failed to process select> Apr 29 01:41:18 localhost service.sh[124976]: java.lang.OutOfMemoryError: Java heap space Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArray(DirectByteBufferStreamImplV> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readByteArray(DirectByteBufferStreamI> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readByteArray(DirectMessageReader.java:189) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.CacheObjectAdapter.readFrom(CacheObjectAdapter.java:145) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateRequest.readFrom(> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:271) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerif> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3734) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNio> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2273) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) Apr 29 01:41:18 localhost service.sh[124976]: at java.base/java.lang.Thread.run(Thread.java:833) Apr 29 01:41:18 localhost service.sh[124976]: [01:41:18,952][SEVERE][grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%][TcpCommunicationSpi] Closing NIO session beca> Apr 29 01:41:18 localhost service.sh[124976]: class org.apache.ignite.internal.util.nio.GridNioException: Java heap space Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNio> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2273) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) Apr 29 01:41:18 localhost service.sh[124976]: at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.lang.OutOfMemoryError: Java heap space Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArray(DirectByteBufferStreamImplV> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readByteArray(DirectByteBufferStreamI> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readByteArray(DirectMessageReader.java:189) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.CacheObjectAdapter.readFrom(CacheObjectAdapter.java:145) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateRequest.readFrom(> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:271) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerif> Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3734) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379) Apr 29 01:41:18 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNio> Apr 29 01:41:18 localhost service.sh[124976]: ... 4 more

[10:50:11,093][SEVERE][sys-stripe-7-#8][GridDhtAtomicCache] <OrderInfo_undeliveredMessages> Unexpected exception during ca> Apr 28 10:50:11 localhost service.sh[58125]: java.lang.OutOfMemoryError: Java heap space Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBina> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAda> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.ja> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.ja> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.ja> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:424) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:4220) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:376) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:6216) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:349) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:6202) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:415) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6403) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:2187) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:2116) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCach> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCache> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridC> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1975) Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomi> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(G> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(Gr> Apr 28 10:50:11 localhost service.sh[58125]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRe> Apr 28 10:50:11 localhost service.sh[58125]: [10:50:11,110][SEVERE][sys-stripe-7-#8][GridCacheIoManager] Failed processing message [senderId=3f7a9f39-a6be-477f-8597-7f>

^-- Baseline [id=0, size=1, online=1, offline=0] Apr 29 01:23:27 localhost service.sh[124976]: [01:23:27] Topology snapshot [ver=32, locNode=33690d9e, servers=1, clients=11, state=ACTIVE, CPUs=36, offheap=41.0GB, hea> Apr 29 01:23:27 localhost service.sh[124976]: [01:23:27] ^-- Baseline [id=0, size=1, online=1, offline=0] Apr 29 01:41:17 localhost service.sh[124976]: [01:41:17,950][SEVERE][grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%][GridDirectParser] Failed to read message [msg> Apr 29 01:41:17 localhost service.sh[124976]: java.lang.OutOfMemoryError: Java heap space Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2$1.create(DirectByteBufferStreamImplV2> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArray(DirectByteBufferStreamImplV> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readByteArray(DirectByteBufferStreamI> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readByteArray(DirectMessageReader.java:189) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.CacheObjectAdapter.readFrom(CacheObjectAdapter.java:145) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateRequest.readFrom(> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImp> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:336) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:271) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerif> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3734) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNio> Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2273) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910) Apr 29 01:41:17 localhost service.sh[124976]: at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) Apr 29 01:41:17 localhost service.sh[124976]: at java.base/java.lang.Thread.run(Thread.java:833)


Solution

  • JVM doesn't have enough free memory and most likely garbage collection is also not able to release enough RAM for reuse. What is your ignite version? How many heap memory you have?

    You should analyse GC logs and take a heap dump. If you version is quite old and there were a memory leak, most likely it was fixed in modern versions.

    https://ignite.apache.org/docs/latest/perf-and-troubleshooting/troubleshooting#debugging-gc-issues