JBoss JMS MDB is losing messages

I have a pretty standard web app on EAP 7.2. Requests come in through a servlet which sends a message to a JMS queue which is then processed by an MDB. Maybe 50% of the time the onMessage() method on the MDB is never invoked when the producer sends the message. There are no errors in the server log file. Here's the basic setup:

jboss config:

<jms-queue name="HIFWebHookQueue" entries="HifWebHookQueue java:jboss/exported/jms/queue/HifWebHookQueue"/>

Servlet side:

@WebServlet(name = "/")
public class MyServlet extends HttpServlet {

    @Inject
    private JMSContext context;

    @Resource(lookup = "java:jboss/exported/jms/queue/HifWebHookQueue")
    private Queue queue;

    protected void doGet(HttpServletRequest request, HttpServletResponse response) {
     
        JMSProducer producer = context.createProducer();
        producer.setDeliveryMode(DeliveryMode.PERSISTENT);
        ObjectMessage msg = context.createObjectMessage(evt);  //evt is a serializable pojo
        producer.send(queue,evt);
    }

the MDB side:

@MessageDriven(name = "WebhookListenerEJB", activationConfig = {
    @ActivationConfigProperty(propertyName="messagingType", propertyValue="javax.jms.MessageListener"),
    @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
    @ActivationConfigProperty(propertyName="destination", propertyValue="java:/jms/queue/HIFWebHookQueue"),
    @ActivationConfigProperty(propertyName="ConnectionFactoryName", propertyValue="ConnectionFactory"),
})
@TransactionManagement(value = TransactionManagementType.CONTAINER)
@TransactionAttribute(value = TransactionAttributeType.REQUIRED)
public class WebhookListenerEJB implements MessageListener {

    public void onMessage(Message message) {
        ObjectMessage msg = (ObjectMessage) message;
        ... stuff ...
        message.acknowledge();
    }
}

Here's output from the JBoss CLI after sending 1st message (and no MDB event):

[standalone@localhost:9990 /] /subsystem=messaging-activemq/server=default/jms-queue=HIFWebHookQueue:read-resource(include-runtime=true)
{
    "outcome" => "success",
    "result" => {
        "consumer-count" => 30,
        "dead-letter-address" => "jms.queue.DLQ",
        "delivering-count" => 0,
        "durable" => true,
        "entries" => [
            "HifWebHookQueue",
            "java:jboss/exported/jms/queue/HifWebHookQueue"
        ],
        "expiry-address" => "jms.queue.ExpiryQueue",
        "legacy-entries" => undefined,
        "message-count" => 0L,
        "messages-added" => 1L,
        "paused" => false,
        "queue-address" => "jms.queue.HIFWebHookQueue",
        "scheduled-count" => 0L,
        "selector" => undefined,
        "temporary" => false
    }
}

Solution

The data you provided indicates that one of the 30 total consumers on HifWebHookQueue received the one message which was sent because messages-added is 1 and message-count (i.e. how many messages are currently on the queue waiting to be consumed) is 0. If the message wasn't consumed I would expect to see message-count and messages-added and consumer-count > 0. The default number of concurrent sessions for an MDB is 15 so I would expect your consumer-count to be 15. Since it is 30 then you almost certainly have another MDB deployed also consuming from HifWebHookQueue.

You can run the following command to see all the consumers for HifWebHookQueue:

/subsystem=messaging-activemq/server=default/jms-queue=HIFWebHookQueue:list-consumers-as-json?