My ActiveMQ Artemis on WidlFly was working fine and some how this week since Monday I started having this exception quite frequently.
At first I went to the stack trace and looked into the code through code grep that throws this exception, and it seemed to me that the issue might be with the large size of the message. I stopped the process that inputs the message in the queues. and turned on my WildFly. The queue did not have data but I still got this exception.
I removed the ActiveMQ data directory and the tx-object-store from the data directory. I started WildFly and it had the same issue, and I am still getting this exception. There was one site saying that may be the ports are wrongly defined, but I have been using this config for months and I don't seem to have changed anything in there nor have I upgraded WildFly 10 to another version so the libs are the same still.
Can any one tell me why is this error coming if there is no data flowing through ActiveMQ. I removed the data directory so there was nothing that is loaded for it to decode nor it is receiving anything to put on the queue to be decoded.
I turned up the log level in ActiveMQ Artemis and the exception looks like this, what is it getting in this log. I am not sending it any messages to process yet it is still processing something.
The issue seems to be from Netty, but I can't seem to point where my config would be wrong esp when I dont think they were changed anywhere.
2016-12-15 15:58:24,707 TRACE [org.apache.activemq.artemis.core.client] (Thread-18 (**activemq-netty-threads**-969652671)) handling packet PACKET(SessionCommitMessage)[type=43, channelID=12, packetObject=SessionCommitMessage]
2016-12-15 15:58:24,707 DEBUG [org.apache.activemq.artemis.core.client] (Thread-18 (activemq-netty-threads-969652671)) Invocation of interceptor org.apache.activemq.artemis.core.protocol.hornetq.HQPropertiesConversionInterceptor on PACKET(SessionCommitMessage)[type=43, channelID=12, packetObject=SessionCommitMessage] returned true
2016-12-15 15:58:24,707 TRACE [org.apache.activemq.artemis.core.server] (Thread-18 (activemq-netty-threads-969652671)) ServerSessionPacketHandler::handlePacket,PACKET(SessionCommitMessage)[type=43, channelID=12, packetObject=SessionCommitMessage]
2016-12-15 15:58:24,707 TRACE [org.apache.activemq.artemis.core.server] (Thread-18 (activemq-netty-threads-969652671)) Calling commit
2016-12-15 15:58:24,707 TRACE [org.apache.activemq.artemis.core.server] (Thread-18 (activemq-netty-threads-969652671)) TransactionImpl::commit::TransactionImpl [xid=null, id=31125, xid=null, state=ACTIVE, createTime=1481781503706(Thu Dec 15 15:58:23 AEST 2016), timeoutSeconds=300, nr operations = 0]@3cd1f6f3
2016-12-15 15:58:24,708 TRACE [org.apache.activemq.artemis.core.server] (Thread-18 (activemq-netty-threads-969652671)) ServerSessionPacketHandler::scheduling response::PACKET(NullResponseMessage)[type=21, channelID=0, packetObject=NullResponseMessage]
2016-12-15 15:58:24,708 DEBUG [org.apache.activemq.artemis.core.client] (Thread-18 (activemq-netty-threads-969652671)) Invocation of interceptor org.apache.activemq.artemis.core.protocol.hornetq.HQPropertiesConversionInterceptor on PACKET(NullResponseMessage)[type=21, channelID=0, packetObject=NullResponseMessage] returned true
2016-12-15 15:58:24,708 TRACE [org.apache.activemq.artemis.core.client] (Thread-18 (activemq-netty-threads-969652671)) Sending packet nonblocking PACKET(NullResponseMessage)[type=21, channelID=12, packetObject=NullResponseMessage] on channeID=12
2016-12-15 15:58:24,708 TRACE [org.apache.activemq.artemis.core.client] (Thread-18 (activemq-netty-threads-969652671)) Writing buffer for channelID=12
2016-12-15 15:58:24,864 TRACE [org.apache.activemq.artemis.core.server] (Thread-72 (activemq-netty-threads-969652671)) Connection created org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@13a92998[local= /192.168.78.30:7045, remote=/10.10.16.11:47709]
2016-12-15 15:58:24,865 ERROR [org.apache.activemq.artemis.core.client] (Thread-72 (activemq-netty-threads-969652671)) AMQ214013: Failed to decode packet: java.lang.IllegalArgumentException: AMQ119032: Invalid type: 1
at org.apache.activemq.artemis.core.protocol.core.impl.PacketDecoder.decode(PacketDecoder.java:413)
at org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:60)
at org.apache.activemq.artemis.core.protocol.ServerPacketDecoder.decode(ServerPacketDecoder.java:202)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:324)
at org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:605)
at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at io.netty.handler.codec.ByteToMessageDecoder.handlerRemoved(ByteToMessageDecoder.java:216)
at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:527)
at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved(DefaultChannelPipeline.java:521)
at io.netty.channel.DefaultChannelPipeline.remove0(DefaultChannelPipeline.java:351)
at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:322)
at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:299)
at org.apache.activemq.artemis.core.protocol.ProtocolHandler$ProtocolDecoder.decode(ProtocolHandler.java:174)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:349)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)
at org.apache.activemq.artemis.core.protocol.ProtocolHandler$ProtocolDecoder.channelRead(ProtocolHandler.java:117)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
My standalone.xml:
<subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">
<server name="default">
<security enabled="false"/>
<management jmx-enabled="true"/>
<journal file-size="1024000"/>
<statistics enabled="true"/>
<shared-store-master/>
<security-setting name="#">
<role name="guest" delete-non-durable-queue="true" create-non-durable-queue="true" consume="true" send="true"/>
</security-setting>
<address-setting name="#" message-counter-history-day-limit="10" page-size-bytes="4194304" max-size-bytes="20971520" expiry-address="jms.queue.ExpiryQueue" dead-letter-address="jms.queue.DLQ"/>
<http-connector name="http-connector" endpoint="http-acceptor" socket-binding="http"/>
<http-connector name="http-connector-throughput" endpoint="http-acceptor-throughput" socket-binding="http">
<param name="batch-delay" value="50"/>
</http-connector>
<remote-connector name="netty" socket-binding="messaging"/>
<remote-connector name="netty-throughput" socket-binding="messaging-throughput">
<param name="batch-delay" value="50"/>
</remote-connector>
<in-vm-connector name="in-vm" server-id="0"/>
<http-acceptor name="http-acceptor" http-listener="default"/>
<http-acceptor name="http-acceptor-throughput" http-listener="default">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</http-acceptor>
<remote-acceptor name="netty" socket-binding="messaging"/>
<remote-acceptor name="netty-throughput" socket-binding="messaging-throughput">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</remote-acceptor>
<in-vm-acceptor name="in-vm" server-id="0"/>
<divert name="divert-to-multidm" forwarding-address="jms.queue.dbservice.multi_dm" address="jms.queue.dbservice.cap_tp_2"/>
<divert name="divert-to-multifo" forwarding-address="jma.queue.dbservice.multi_fo" address="jms.queue.dbservice.cl1test_odc_capfotp2"/>
<!--divert name="divert-to-cl1fo" forwarding-address="jma.queue.dbservice.cl1_fo" address="jms.queue.dbservice.cl1test_odc_capfotp2"/>
<divert name="divert-to-tmafocl1devfo" forwarding-address="jma.queue.dbservice.tmafocl1dev_fo" address="jms.queue.dbservice.cl1_test_odc_capfotp"/>
<divert name="divert-to-tmafocl1devdm" forwarding-address="jma.queue.dbservice.tmafocl1dev_dm" address="jms.queue.dbservice.cl1_test_odc_cap_tp"/-->
<jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
<jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
<jms-queue name="dbservice.cap_tp_2" entries="/queue/cap_tp_2"/>
<jms-queue name="dbservice.tp2_dm" entries="/queue/tp2_dm"/>
<jms-queue name="dbservice.tp2_fo" entries="/queue/tp2_fo"/>
<!--jms-queue name="dbservice.tmafocl1dev_fo" entries="/queue/tmafocl1dev_fo"/>
<jms-queue name="dbservice.tmafocl1dev_dm" entries="/queue/tmafocl1dev_dm"/-->
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<jms-queue name="dbservice.tmafocl1dev_dm" entries="/queue/tmafocl1dev_dm"/-->
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector" />
<pooled-connection-factory name="hornetq-ra" transaction="xa" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" min-large-message-size="10240"/>
</server>
</subsystem>
...
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:9443}"/>
<socket-binding name="iiop" interface="unsecure" port="3528"/>
<socket-binding name="iiop-ssl" interface="unsecure" port="3529"/>
<socket-binding name="messaging" port="5445"/>
<socket-binding name="messaging-backup" port="5545"/>
<socket-binding name="messaging-throughput" port="5455"/>
<socket-binding name="txn-recovery-environment" port="4712"/>
<socket-binding name="txn-status-manager" port="4713"/>
<socket-binding name="node2-jms-broker" port="${node2.broker.port:5445}"/>
<socket-binding name="node2throughput-jms-broker" port="${node2throughput.broker.port:5455}"/>
<outbound-socket-binding name="mail-smtp">
<remote-destination host="localhost" port="25"/>
</outbound-socket-binding>
</socket-binding-group>
The problem was that My ActiveNMQ queues were only write only queues, that is my application only put data on the queues and we never expected that there would be any data put onto them. our active MQ server was uni directional, just putting messages on queues for other apps to consume.
What happened was that in trying something, a developer on the consuming end created an ActiveMQ server instead of creating a client to read from the queues in their glassfish and I think they were pushing some message. I should have viewed the message in the queue but resolved the issue before that. so I had one queue that had a server on both ends ( somehow) the activeMQ on the other end was putting some messages on it i think. we removed the configuration from glassfish for the activeMQ and it was fine.
Apologies for answering so late. should have posted this for others who would setup a server trying to put messages on a consuming queue, the producer will throw this error.