I've used the heck out of storage queues, typically with Azure queue-triggered functions and no visibility delay. I have recently encountered a use case where I need to schedule events, sometimes 90 days out. I know that the TTL for storage queues can now be unlimited, but it seems that the visibility delay is still capped at seven days.
Setting aside the option of using the Azure Service Bus or building a durable queue with a database table, and just focusing on Azure storage queues, is there a well-established pattern for getting more life out of a storage queue message than seven days?
I can imagine dequeueing the message, moving the visibility delay out another seven days, and just continuing to do this until the target time is within seven days, whereupon the visibility delay would be set precisely. I don't know if I can modify the visibility delay property and re-enqueue the same message, or if I would have to create a new message and delete the 7-day old one. Edit: it seems like the former is the proper way.
Wondering if anyone has a known good solution to this, or if you would wave me off this kind of approach entirely. Is there a role for a durable function here with a timer instead of using my queue directly? It seems like PR 1390 implemented a similar "grab and resubmit" process for durable timers. So on one hand, I'm wondering if durable functions are a more proper solution for my scenario, but on the other hand it seems like the strategy used to enable this long-running timer in durable functions is exactly the kind of loop I could do with a queue-triggered function.
Any advice on the best approach would be appreciated, particularly any guidance on implementation if you have done this before and think it's viable.
I have tried changing the service version for the QueueClient up to 2021_10_04, and messages submitted with a visibility timespan of > 604k seconds (7 days) crap out with a 400 bad request exception. Under 604k seconds, the message is accepted into the queue. All messages are being submitted with a TTL of -1 (no expiration). I'm testing this using Azurite 3.19.0 and latest build of VS 2022 locally. My Azure.Storage.Queues library is at v12.12.0. .Net6 if it matters. I guess I expected that I could extend the visibility timespan since there's no cap on TTL anymore, but it doesn't seem to be the case.
Thank you for any help/advice.
Well, hearing none, seeing none, I'll update this with the route I took and will come back and post more as time provides me with more insight into my approach.
Basically, my process for placing messages into the queue serializes an event object which contains a schedule date. If the schedule date is further out than 7 days, I use the max visibility delay of seven days. If not, I use the precise visibility delay to match the schedule date.
My queue-triggered function will grab a message when visible, and after deserializing the message body, will check if the schedule date is 'ripe'. If not, it repeats the same evaluation to derive a new visibility delay and then updates the message with a refreshed visibility delay. If the date is 'ripe' -- and I bake in a bit of a fudge factor in evaluating this -- I act on the message (which then gets deleted from queue).
It seems to work fine. Time will tell me how robust it is.