Mule 4 anypoint mq subscriber - circuit breaker not working

Am trying to implement a circuit breaker pattern . Using Mule 4.4. Am consuming a message from Anypoint MQ queue and then trying to call a downstream api

here is the code :

<flow name="create-emp-flow">
        <anypoint-mq:subscriber doc:name="Subscriber - Create user" 
        config-ref="Anypoint_MQ_Configuration" destination="emp.create.queue}" 
        acknowledgementMode="AUTO">
            <redelivery-policy maxRedeliveryCount="1"/>
            <anypoint-mq:circuit-breaker tripTimeout="150000"
            onErrorTypes="HTTP:INTERNAL_SERVER_ERROR,MULE:UNKNOWN" errorsThreshold="3"/>
        </anypoint-mq:subscriber>
        
        
        <http:request method="GET" doc:name="HACK - Mock CALL return a ISE500" 
        config-ref="MOCK_Request_Configuration" path="/emp" >
                <http:response-validator >
                    <http:success-status-code-validator values="200" />
                </http:response-validator>
            </http:request>
            
        
        <error-handler >
            <on-error-continue enableNotifications="true" logException="true" 
            doc:name="On Error Continue" type="MULE:REDELIVERY_EXHAUSTED">
               <anypoint-mq:publish doc:name="Publish to DLQ"  
               config-ref="Anypoint_MQ_Configuration" destination="emp.dlq" />              
            </on-error-continue>
            
            <on-error-propagate enableNotifications="true" logException="true" 
            doc:name="On Error Propagate"  >
             <logger .../>
            </on-error-propagate>
        </error-handler>
</flow>

So the sequence is : #1 consume message from queue ( maxRedeliveryCount=1 ) #2 then call a HTTP endpoint - here I am purposefully returniing a 500 ISE error

Note - the subscriber is configured with a Circuit breaker with a maxThreshhold = 3 and types of error to trigger Circuit breaker being HTTP:INTERNAL_SERVER_ERROR,MULE:UNKNOWN

#3 Now since the HTTP call is returning an ISE the exception in mule shows as HTTP:INTERNAL_SERVER_ERROR which is what I would like to generate

Now since I have redelivery configured the message is retried once more and again on failure we reach the condition MULE:REDELIVERY_EXHAUSTED , so message is published to DLQ

Things are good upto this point . Now I publish multiple messages to the queue and same behaviour as above is observed

Now I would expect circuit breaker to kick in after the 3rd message is consumed , retried and published to DLQ . After the 3rd message was expecting the subsequent messages to NOT be consumed till 2.5 minutes have elapsed

However the subscriber seems to merrily consume messages with no sign of Circuit breaker kicking in .

Note - I want to use the <redelivery-policy maxRedeliveryCount="1"/> and not rely on max delivery which is configured on the queue in Anypointmq ( 10 by default )

so not sure why circuit breaker is not kicking in ?

From the Connector documentation

If a Mule flow finishes its execution with an error, the Subscriber source checks if the error is one of the onErrorTypes that indicates an external service error, and counts consecutive occurrences until errorsThreshold is reached.

So the errors should be consecutive to transition the circuit in "open" state. The problem with your implementation is that you have on-error-continue for REDELIVERY_EXHAUSTED error, which means that the subscriber will treat the execution of the poll as a success and not an error. And therefore it will stop the circuit to get into open state and the error counter will reset

For making it work with the redelivery-policy you need to make the following changes

1. Change on-error-continue to on-error-propogate
2. Configure the subscriber to use Manual acknowledgement mode instead of auto
3. Add a ACK in the on-error-propogate handler for the REDELIVERY_EXHAUSTED error to delete the message from MQ
4. Add ACK at the end of the flow to delete the message from MQ
5. Add NACK in the default on-error-propogate to retain the message for the next poll

This will make the source to not consider the REDELIVERY_EXHAUSTED as a success and the circuit should open after consecutive 3 errors (which is the errorThreshold)