I have a java program which keeps calling java.util.zip to compress/decompress data. It runs out of memory within a few seconds.
I had a memory dump with jmap and I'm viewing it with jhat.
Finalizer summary shows Total instances pending finalization: 0. If I understand it correctly, I don't have any objects which (1) have finalize() method, (2) have been marked by GC and (3) are waiting to be finalized. This seems good.
When I look at a particular object, the only reference to this object is a java.lang.ref.Finalizer. The Finalizer object is created for each object which has a finalize() method no matter if the object is GC'ed or not. So it looks like nothing is preventing this this Deflater object to be GC'ed.
Object at 0x7f4aeb7a35d0
instance of java.util.zip.Deflater@0x7f4aeb7a35d0 (51 bytes)
References to this object:
java.lang.ref.Finalizer@0x7f4aeb8607c8 (64 bytes) : field referent
The program is paused in the run by System.in.read(). The memory usage doesn't go down after a while.
UPDATE:
I should make it clear. The memory dump shows many objects were not GC'ed but no other objects (except Finalizer objects) were referencing them. I'm trying to find out why they were not GC'ed.
Are you positive you closed the stream and invoked end on the deflater? I'm sorry if this is a simplistic suggestion you've already tried, but there were a lot of complaints about a memory leak when using Deflater when not immediately invoking end on it, for example:
The root cause is apparently that the collector is not able to keep up with the application when memory used by native elements is involved. That also explains the behavior you are seeing when profiling: the memory is ready to be reclaimed, but just isn't reclaimed quickly enough.
Since you wrote you don't use a Deflator directly but through Apache Thrift, try to identify which method in that library is responsible for ending the deflator, and made sure you invoked that method.