duplicateshigh-availabilityfailoveractivemq-artemisfailovercluster

Artemis receives duplicate messages after failover


In order to test the communication performance in the event of a failure, I numbered each message and sent it continuously, sending about 30 messages per second. And found that even if the ha policy is set, consumers will repeatedly receive a small number of received messages after failover/failback. Is this normal?

I know that Artemis provides automatic duplicate message detection by giving a unique value to the message, which can avoid repeated sending of messages, but the repeated received messages have different "client ack messageID". Does this mean that it cannot prevent receiving repeated messages?


Solution

  • Depending on how you've written your client you can get duplicates on failover because some message acknowledgements may get lost when the failure happens. For example, if you receive a message from the broker and process it but then the broker fails before you send the acknowledgement (or fails while the acknowledgement is in transit) then the backup will still have the message you received already and will dispatch it again.

    If you don't want duplicates to be a problem for your client then you have a couple of options:

    1. Use a transaction on your client and don't commit until the acknowledgement has been confirmed successfully. If the acknowledgement fails then rollback the transaction.
    2. Make sure your consumer is idempotent so duplicates don't really matter.