Reliable Asynchronous Handling of Domain Events

In concurrent systems, domain events are typically handled asynchronously. In Go, a simple approach for asynchronous event handling can be implemented via channels, but the issue is that if something bad happens for handling an event, or worst, for the whole program, the event will be lost.

How asynchronous domain events can be handled properly in a Go program, i.e.:

• When an event handler fails, the event should not be purged from the event queue, in order to be handled properly in a later time.
• If the whole program goes down, the events have to be recovered and processed accordingly.

The first is relatively easy; you can have an error handler within the worker that re-queues the work in the event of an error.

The second is much harder; your options are a) roll your own bulletproof mechanism for writing events to disk and purging them when they're completed in a thread-safe way or b) use one of the many, many popular systems available that's already proven reliable, e.g. RabbitMQ or Kafka, with the appropriate replication and redundancy to ensure the level of reliability you require. I would strongly recommend the latter.