Given that I have a service P (Producer) and C (Consumer) my P service needs to:
From C's perspective, is there a way to guarantee that it had all the info it needed for successful persistence?
I know that services like Kafka and Redis Streams are not generally built for this stuff, but how does one overcome this?
Idea 1:
Idea 2:
Is there any obvious way to handle object dependencies?
I believe IDEA 2 is a better solution cuz I think keep the whole message in one data structure is a good idea.
And probably you can try to use multiple keys.
For example:
def now_timestamp = datetime.currentstamp # let`s say it is 1515151551
redis sadd not_processed_timestamp 1515151551
redis set X_1515151551 INFO_OF_X
redis set Y_1515151551 INFO_OF_Y
redis set Z_1515151551 INFO_OF_Z
def new_task_timestamp = redis spop not_processed_timestamp # let`s say it is 1515151551
redis blocking-get X_1515151551
redis blocking-get Y_1515151551
redis blocking-get Z_1515151551
# process the rest