apache-kafkaaeron

How to replace Kafka with Aeron


At present, the trading system of our production environment is using Kafka. Because Kafka latency is too high, we hope to replace Kafka with Aeron. How can I use Aeron correctly?


Solution

  • Aeron isn't an out of the box replacement for Kafka although it does provide primitives that would allow you to replicate much of the functionality.

    Kafka latencies are in the order of milliseconds whereas Aeron latencies are typically measured in microseconds.

    What exactly you would need to build in Aeron very much depends on your use case.

    One of the primary uses of Kafka is as a persistent queue.

    To build a simple persistent queue for a single publisher use case. You would need:

    Publisher

    Subsciber(s)

    There are examples of this in the aeron-samples.

    Latency could be reduced further by having the publisher send messages over multicast/MDC and having the subscriber use ReplayMerge to seamlessly transition from the recorded stream to the live stream.

    Worth noting that real-logic do provide commercial support.