javaapacheglassfishload-balancingajp

org.glassfish.grizzly.http.ajp.AjpHandlerFilter.encodeHttpPacket(AjpHandlerFilter.java:282) when using Apache as load balancer for GF 5


I've configured a Glassfish 5.0 instance with an AJP connector that would work as load balancer even though at the moment I have only one glassfish.

In domain.xml I have the followin:

<http-service>
    <access-log></access-log>
    <virtual-server network-listeners="http-listener-1,http-listener-2,jk-connector" id="server"></virtual-server>
    <virtual-server network-listeners="admin-listener" id="__asadmin"></virtual-server>
</http-service>

where jk-connector is the one I use from the load balancer.

 <network-config>
     ...
      <protocols>
          <protocol name="jk-connector">
            <http default-virtual-server="server">
              <file-cache></file-cache>
            </http>
          </protocol>
        </protocols>
    </network-config>

then there is the listener

    <network-listeners>
      <network-listener protocol="jk-connector" jk-enabled="true" port="8009" name="jk-connector" thread-pool="http-thread-pool" transport="tcp"></network-listener>
    </network-listeners>

My Apache http.conf is as follows:

LoadModule headers_module modules/mod_headers.so
LoadModule jk_module modules/mod_jk.so

# Where to find workers.properties
JkWorkersFile conf/workers.properties
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"

<IfModule mod_rewrite.c>
        Options +Indexes
        Options +FollowSymlinks
        RewriteEngine on
        RewriteMap  uc int:toupper

        #rewrite per compatibilita' con vecchie URL dei servizi 
        RewriteRule ^/services/(.)(.*)$   /api/${uc:$1}$2Services?%{QUERY_STRING}  [PT]
        #RewriteRule ^/services/(.)(.*)$  http://%{HTTP_HOST}/api/${uc:$1}$2Services?%{QUERY_STRING}  [R=301,L]

        #solo nel caso di un GET
        RewriteCond  %{REQUEST_METHOD}  GET
        ReWriteRule ^(.*)/login.jsf;jsessionid=[A-Za-z0-9\.]+(.*)$ $1/login.jsf$2 [L,R=301]
</IfModule>

The exception is:

  GRIZZLY0013: Exception during FilterChain execution
java.lang.NullPointerException
    at org.glassfish.grizzly.http.ajp.AjpHandlerFilter.encodeHttpPacket(AjpHandlerFilter.java:282)
    at org.glassfish.grizzly.http.ajp.AjpHandlerFilter.handleWrite(AjpHandlerFilter.java:245)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:890)
    at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:858)
    at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1092)
    at org.glassfish.grizzly.http.io.OutputBuffer.flushBinaryBuffers(OutputBuffer.java:1066)
    at org.glassfish.grizzly.http.io.OutputBuffer.flushAllBuffers(OutputBuffer.java:1036)
    at org.glassfish.grizzly.http.io.OutputBuffer.close(OutputBuffer.java:754)
    at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:244)
    at org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:179)
    at java.nio.channels.Channels$WritableByteChannelImpl.implCloseChannel(Channels.java:469)
    at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
    at com.sun.faces.application.resource.ResourceHandlerImpl.handleResourceRequest(ResourceHandlerImpl.java:365)
    at javax.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:178)
    at org.primefaces.application.resource.PrimeResourceHandler.handleResourceRequest(PrimeResourceHandler.java:87)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:667)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1580)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:338)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:100)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:250)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at com.ocpsoft.pretty.PrettyFilter.doFilter(PrettyFilter.java:145)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:250)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:250)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:652)
    at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:591)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:463)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:168)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:206)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:180)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:242)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:539)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:593)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:573)
    at java.lang.Thread.run(Thread.java:748)

Using JDK 1.8.0_131, Glassfish 5.0, Apache 2.4.2


Solution

  • Seems like you are facing this Glassfish bug (and this one).

    You can try the latest Glassfish 5 build which is available here: http://download.oracle.com/glassfish/5.0.1/nightly

    Unfortunately it looks like there is currently no solution besides manually fixing the grizzly JAR file.

    You can try the following:

    1. Download the grizzly 2.4 sources (https://github.com/javaee/grizzly/archive/2.4.x.zip) and extract them
    2. Open the project grizzly-http-ajp and build it, if the build fails due to missing dependencies, change the project version to 2.4.3 in pom.xml
    3. If it compiles without problems, open the class AjpHandlerFilter
    4. Do the following change near line 276 so the code looks like this:

          if (contentBuffer.hasRemaining()) {
              return AjpMessageUtils.appendContentAndTrim(memoryManager,
                      encodedBuffer, contentBuffer);
          } else if (encodedBuffer == null) {
              encodedBuffer = Buffers.EMPTY_BUFFER;
          }
      
    5. Save and compile the project

    6. Go to /grizzly-2.4.x/modules/http-ajp/target/classes/org/glassfish/grizzly/http/ajp/ and copy the the generated .class file for AjpHandlerFilter
    7. Update the AjpHandlerFilter.class in the glassfish-grizzly-extras-all.jar (inside glassfish/glassfish/modules) of your Glassfish installation. You can do this with jar uf ${jar-file} ${class-file}. You need to make sure that the path is correct and you that you replace the old .class file in the JAR

    Alternativly you can try to find the sources of the project glassfish-grizzly-extras-all, fix the problem there and build a new glassfish-grizzly-extras-all.jar.