[SOLVED] how to set the consumer in Faust to a specific offset

how to set the consumer in Faust to a specific offset

From the Faust documentation I can't find out how to set the consumer to a specific offset.

With confluent-kafka I use consumer.offsets_for_times to find a start_offset and then assign the TopicPartition to that specific offset, something like:

start_offset = consumer.offsets_for_times([
    TopicPartition("prediction.OfferPredictionCheckpoint", 0, int(start_date)),
    TopicPartition("prediction.OfferPredictionCheckpoint", 1, int(start_date)),
])

consumer.assign([
    TopicPartition("prediction.OfferPredictionCheckpoint", partition_number, pos)
])

With Faust I can't find much more than:

consumer_auto_offset_reset

Which only let you set earliest or latest. How would I start reading from a specific hour or beginning of day?

Solution

I think this might be what you're looking for: https://faust.readthedocs.io/en/latest/reference/faust.transport.consumer.html#faust.transport.consumer.Consumer.seek

It can go to a specific offset, however I don't think there's an easy way to tell it to go to a specific time or date without some extra logic (maybe binary search your way there using the offset?).