I make a service which reads a recording from Aeron Archive and sends this data to Kafka. The recordings operate with position: they have startPostion/stopPosition. When you replay the data you need to specify the position, etc. If the Archive consumer restarts, then it wants to continue processing from the point where it stopped last time. How should it be implemented? There are two subquestions here:
When I receive a fragment in a FragmentHandler, how do I figure out which position does it have in the recoding? The Subscription/FragmentHandler API is implemented for the basic Aeron which have no information about the Archive.
More generally, how should consumers track their offsets? Kafka provides this functionality: when a consumer from a given consumer group connects to Kafka, it can start reading from where it stopped last time without storing the consumer offset on a client side. Does Aeron Archive have anything like this? What is the expected way of tacking offsets?
in the Aeron Archive API you can specify start position in the parameters for startReplay.
When I receive a fragment in a FragmentHandler, how do I figure out which position does it have in the recoding?
when you receive a fragment in your FragmentHandler it must have the same position as in the original stream and the same position as in the recording on the disk (in Aeron Archive).
how should consumers track their offsets?
the position itself you can get from io.aeron.logbuffer.Header#position
or you can check what was the final position of a recording with method io.aeron.archive.client.AeronArchive#getStopPosition (note: if a recording is live it will return -1, for current position you may use io.aeron.archive.client.AeronArchive#getRecordingPosition)
I can recommend to take a look at this test https://github.com/real-logic/aeron/blob/master/aeron-system-tests/src/test/java/io/aeron/archive/BasicArchiveTest.java#L624