javamemory-leaksgarbage-collectionclassloaderglassfish-2.x

Classloader is not garbage collected, even without GC roots

We have a complex application running under Glassfish V2.1.1. In order to be able to load our code dynamically, we have implemented a CustomClassloader which is able to redefine classes. The behaviour is quite easy: when a dynamically loaded class has change, the current instance of the CustomClassloader is "dropped" and a new one is create to redefine the needed classes.

This works well except that after a few number of time the same class has been reloaded (hence each time a new CustomClassloader is created), we get a PermGen space error because the other instances of CustomClassloader are not garbage collected. (There should be only one single instance of this class)

I tried different methods to track down where the leak is:

  1. visualvm => I make a heap dump and extract all the instances of the CustomClassloader. I can see that none of them have been finalized. When I check for the nearest GC root, visualvm tells me that there is none (except for the last instance, because it is the 'real' used one).
  2. jmap/jhat => It gives me almost the same result: I see all the instances of CustomClassloader and then when I click on the link to see where are the references on one of them, I get a blank page meaning there is none...
  3. Eclipse Memory Analyzer Tool => I get a strange result when I run the following OQL query: SELECT c FROM INSTANCEOF my.package.CustomClassloader c There is only one result, indicating there is only one single instance which is obviously not correct.

I also checked this link and implemented some resources release when a new CustomClassloader is created, but nothing changes: the PermGen memory is still increasing.

So I'm probably missing something and the difference between points (1-2) and (3) shows something I do not understand. Where can I look to get an idea of what's wrong ? Since all the tutorials I followed show how to search the leaking references by using the "Search nearest GC root" feature (and in my case there is none), I don't know how I can track the error.

EDIT 1: I uploaded an example of the heap dump here. The ClassLoader not being unloaded can be selected in visualvm with the following query: select s from saierp.core.framework.system.SAITaskClassLoader s One can see that there are 4 instances and the three first should have been collected because there is no GC root... There must be a reference somewhere but I don't know how I can search for it. Any hint is welcomed :)

EDIT 2: After some deeper tests I see a very strange pattern. The leak seems to depend on the data that are being loaded by OpenJPA: if no new data is loaded, then the classloader can be GCed, otherwise it is not. Here is the code I use when a create a new SAITaskClassLoader to 'clear' the old one:

PCRegistry.deRegister(cl);
LogFactory.release(cl);
ResourceBundle.clearCache(cl);
Introspector.flushCaches();

= Pattern 1 (Classloader is GCed): =

  1. New SAITaskClassLoader
  2. Load Data D1, D2, ..., Dn
  3. New SAITaskClassLoader
  4. Load Data D1, D2, ..., Dn
  5. ...

cl-gc

= Pattern 2 (Classloader is NOT GCed): =

  1. New SAITaskClassLoader
  2. Load Data D1, D2, D3
  3. New SAITaskClassLoader
  4. Load Data D3, D4, D5
  5. New SAITaskClassLoader
  6. Load Data D5, D6, D7
  7. ...

cl-nogc

In all cases, the SAITaskClassLoader that have been cleared have no GC root. We are using OpenJPA 1.2.1.

Thanks & Best regards

Solution

  • Finally, I can close this bug, since it seems to be linked with OpenJPA and non-parametrized queries. Another thread to look at: Custom ClassLoader not garbage collected