I am writing a Java instrumentation program that uses the built-in Instrumentation API with Javassist (v3.26.0-GA) to intercept all the method calls in the target program. Also, I have implemented a REST API service inside this program using Java Spark to send requests for starting/stopping instrumentation by adding/removing transformers, and also for fetching intercepted methods during the instrumentation time.
Now, while I was trying to run WebGoat (an open source Spring Boot application) with my Java agent attached from premain, I was not able to intercept all the methods successfully and in the log, there was a NotFoundException being thrown by Javassist.
This error happened for several classes in WebGoat all had a similar common fact that they had something to do with SpringCGLIB. A few of the errors are shown below.
javassist.NotFoundException: org.owasp.webgoat.hijacksession.cas.HijackSessionAuthenticationProvider$$FastClassBySpringCGLIB$$88f1f22d
at javassist.ClassPool.get(ClassPool.java:430)
at com.sparrow.sptracer.core.transformer.AbstractMethodTransformer.transform(AbstractMethodTransformer.java:87)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.defineClass0(Native Method)
at java.base/java.lang.System$2.defineClass(System.java:2307)
at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2439)
at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2416)
at java.base/java.lang.invoke.MethodHandles$Lookup.defineClass(MethodHandles.java:1843)
at java.base/jdk.internal.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:507)
at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:363)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:110)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:108)
at org.springframework.cglib.core.internal.LoadingCache$2.call(LoadingCache.java:54)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:61)
at org.springframework.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:134)
at org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:319)
at org.springframework.cglib.reflect.FastClass$Generator.create(FastClass.java:65)
at org.springframework.cglib.proxy.MethodProxy.helper(MethodProxy.java:135)
at org.springframework.cglib.proxy.MethodProxy.init(MethodProxy.java:76)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:216)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.owasp.webgoat.hijacksession.cas.HijackSessionAuthenticationProvider$$EnhancerBySpringCGLIB$$6ae99c75.authenticate(<generated>)
at org.owasp.webgoat.hijacksession.HijackSessionAssignment.login(HijackSessionAssignment.java:72)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:219)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:213)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:833)
javassist.NotFoundException: org.owasp.webgoat.hijacksession.cas.HijackSessionAuthenticationProvider$$EnhancerBySpringCGLIB$$6ae99c75$$FastClassBySpringCGLIB$$8c045873
at javassist.ClassPool.get(ClassPool.java:430)
at com.sparrow.sptracer.core.transformer.AbstractMethodTransformer.transform(AbstractMethodTransformer.java:87)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.defineClass0(Native Method)
at java.base/java.lang.System$2.defineClass(System.java:2307)
... similar stacktrace as above
javassist.NotFoundException: org.owasp.webgoat.session.UserSessionData$$FastClassBySpringCGLIB$$2b6b54bc
at javassist.ClassPool.get(ClassPool.java:430)
at com.sparrow.sptracer.core.transformer.AbstractMethodTransformer.transform(AbstractMethodTransformer.java:87)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.defineClass0(Native Method)
at java.base/java.lang.System$2.defineClass(System.java:2307)
... similar stacktrace as above
[IMPACTRACER-CORE] ERROR [2021-12-24 10:34:20]: NotFoundException on class 'org/owasp/webgoat/session/UserSessionData$$EnhancerBySpringCGLIB$$bbb61fe2$$FastClassBySpringCGLIB$$34cb52d3': org.owasp.webgoat.session.UserSessionData$$EnhancerBySpringCGLIB$$bbb61fe2$$FastClassBySpringCGLIB$$34cb52d3
javassist.NotFoundException: org.owasp.webgoat.session.UserSessionData$$EnhancerBySpringCGLIB$$bbb61fe2$$FastClassBySpringCGLIB$$34cb52d3
at javassist.ClassPool.get(ClassPool.java:430)
at com.sparrow.sptracer.core.transformer.AbstractMethodTransformer.transform(AbstractMethodTransformer.java:87)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
... similar stacktrace as above
My guess is that WebGoat uses cglib in its Spring Boot environment to dynamically generate bytecode and the corresponding class is not included in the WebGoat's classpath originally, so javassist.ClassPool.get(className) is throwing the error.
My code for creating ClassPool object is as below.
String name = Descriptor.toJavaName(className);
try {
ClassPool cp = ClassPool.getDefault();
cp.childFirstLookup = true;
cp.appendClassPath(new LoaderClassPath(loader));
CtClass cc = cp.get(name);
Logger.debug("Checking class %s", name);
CtMethod[] methods = cc.getDeclaredMethods();
Logger.debug("Altering %d methods in %s", methods.length, wut);
for (CtMethod m : methods) {
// do some code insertion
}
bytecode = cc.toBytecode();
cc.detach();
} catch (NotFoundException e) {
Logger.error("NotFoundException on class '%s': %s", className, e.getMessage());
e.printStackTrace();
} catch (CannotCompileException e) {
Logger.error("Cannot compile class '%s': %s", className, e.getMessage());
e.printStackTrace(System.out);
} catch (IOException e) {
Logger.error("IOException while transforming class '%s': %s", className, e.getMessage());
} catch (Exception ex) {
Logger.error("Generic exception occurred while transforming class '%s': %s", className, ex.getMessage());
ex.printStackTrace(System.out);
}
The above errors occured when I tried interacting with the WebGoat application on localhost:8080 tried to send a "start istrumentation" request with a specific id using Java Spark REST API, so whenever I send this request, it triggers inst.addTransformer method to create a new Transformer and adds it to the instrumentation object.
What could be the cause of this error? I assume that WebGoat itself is using some instrumentation which means I am instrumenting an instrumented application and I don't know if that is even possible.
Any insight would be much appreciated.
From previous comments:
The unfound classes are dynamic proxies which are heavily used by the Spring Framework in order to implement AOP. Spring can use both JDK dynamic interface proxies and CGLIB proxies, the latter of which is what we are seeing here. Maybe you should simply ignore those types of classes. They are in fact created dynamically, hence the name. But they are rather a result of dynamic (sub-)class generation than of bytecode transformation.
Yes, I have considered just ignoring those dynamically generated classes, but the whole point of my application was to capture every single method invocation as a user interacts with the web application (such as clicking on a button, etc). In this case, would it be okay to ignore these types of dynamically generated classes? I want to make sure I do not miss any method calls.
As those classes are just dynamic proxies, they will either forward the calls to the original methods or call some AOP or interceptor logic first/instead. Either way, you would not miss anything essential, those proxies are more like switchboards or routers, the actual show happens somewhere else. I recommend you to simply try in a little playgrounds project with an aspect or two.
You also asked how to detect and ignore dynamic proxies by their names:
CGLIB proxies: Spring's CGLIB proxies contain substrings like $$FastClassBySpringCGLIB$$ or $$EnhancerBySpringCGLIB$$, followed by 8 characters representing 4 hexadecimal bytes. You could either match with a regular expression of just keep it simple and match the substring BySpringCGLIB$$. If non-Spring CGLIB proxies are also in use somewhere in your application, you would have to watch for other naming patterns. But probably you would get similar errors as before when not filtering them, so you would notice automatically.
JDK proxies: If your Spring application also happens to use JDK proxies, you can identify them easily using JRE API call Proxy.isProxyClass(Class). Thanks to Johannes Kuhn for his comment.
JDK proxies (old answer): You can filter class names beginning with $Proxy, usually something like com.sun.proxy.$Proxy2 (the trailing number being different). According to the JDK documentation: "The unqualified name of a proxy class is unspecified. The space of class names that begin with the string "$Proxy" is, however, to be reserved for proxy classes." At least for Oracle and probably OpenJDK, you can match for that naming pattern. If that holds true for all JVMs, is up to you to test, if chances are that in your environments others are being used. I quickly tried with Semeru OpenJ9, and the proxy naming pattern is identical, even the package name com.sun.proxy. Pleasae note that in more recent JDK versions, JDK proxies will have fully qualified names like jdk.proxy2.$Proxy25, so in e.g. Java 16 or 17 you should not rely on package name com.sun.proxy. Either add more cases or limit matching to the leading $Proxy in the simple class name.
Update 2022-02-26: Because there was activity on this question, I decided to add some more information about Spring-specific tools which can determine whether an object (or a class) is an AOP proxy (class) and, more specifically, if it is a CGLIB or JDK proxy:
Take a look at tool class AopUtils and its handy methods
isAopProxy(Object),isCglibProxy(Object),isJdkDynamicProxy(Object).No more String matching, simply ask Spring.
BTW, there is also a method net.sf.cglib.proxy.Proxy.isProxyClass(Class) directly in CGLIB, which is supposed to do the same, but within Spring it does not work, probably because Spring uses CGLIB in a non-canonical way. Because Spring embeds a package-relocated CGLIB in its core, the corresponding method org.springframework.cglib.proxy.Proxy.isProxyClass(Class) yields the same faulty result. So if you are working within Spring, please do not use those methods, better use AopUtils.
Here is some example code for your convenience, showing how to determine Spring AOP proxy types (JDK vs. CGLIB proxies) using AopUtils. See also my answer here for how to configure Spring in order to use both proxy types.
BTW, instead of Javassist you could also use AspectJ for your purpose. It sounds like a pretty typical use case.