azure-eventhub

Does Azure Event Hub guarantees at least once delivery?

I'm building an azure web application, I'd like to send the activity logs to Azure Event Hub. What happen if the connection between the application host and the event hub is lost? Does the Event Hub client implement some kind of local queue?

Solution

  • TLDR: Yes. EventHubs Offers atleast-once delivery. No queue is maintained by EventHub client SDK. It will throw and the dependent App will need to retry the Send.

    EventHubs service guarantees atleast-once delivery. The send call will succeed only when client receives 'acknowledgement" from the EventHubs Service.
    EventHubs service will not wait for the client's 'acknowledgement' for the 'acknowledgement' it has sent. In simple words - it doesn't offer Only-Once/Exactly-once semantics.

    Implementor perspective: If you are familier with the Client SDK, EventHubs Service, do not even, have a way to Know that A message is being sent/delivered twice – because, there is no way currently built into it – to identify A message (for ex: there is nothing like a MessageID in EventData). So, the only guarantee that EventHubs Service can offer is – whatever data is sent to eventHub service – it will acknowledge back to the Client – only-and-only after persisting the message to a persistent store. That’s why, it’s called at least once. Same applies to receive.
    If a receiver client crashes, after recovering - come back with the offset that you last remember – EventHubs service will guarantee that it will replay the stream from that exact point.

    The trade-off call: To offer any other semantics like ‘at-most-once’ or ‘exactly-once’ – message-level infrastructure (an identifier per-message and the Compute per-message to de-duplicate the Events) - will be needed. So, this nice to have feature at the Service level - will come with extra performance overhead.

    As the purview of Event Hubs is to offer Stream-semantics and these are really at per-Message semantics – Event Hubs Service chose to push this to Client libraries.
    Clients libraries will need to build it – taking dependency on - the Exactly-once semantics that is offered.