orleans

Is Orleans reminder execution interleaved?


If there are two different reminders on the same grain activation to be fired at the same point, given that grain execution context is single-threaded, will both reminders be executed and interleaved at the same time?

Also, is the reminder execution limited by the default 30s timeout ?


Solution

  • Reminders are invoked using regular grain method calls: the IRemindable interface is a regular grain interface. IRemindable.ReceiveReminder(...) is not marked as [AlwaysInterleave], so it will only be interleaved if your grain class is marked as [Reentrant].

    In short: no, reminder calls are not interleaved by default.

    Reminders do not override the SiloMessagingOptions.ResponseTimeout value, so the default execution time will be 30s.

    If you have a reminder that might need a very long time to execute, you can follow a pattern of starting the long-running work in a background task and ensuring that it is still running (not completed or faulted) whenever the relevant reminder fires.

    Here is an example of using that pattern:

    public class MyGrain : Grain, IMyGrain
    {
        private readonly CancellationTokenSource _deactivating = new CancellationTokenSource();
        private Task _processQueueTask;
        private IGrainReminder _reminder = null;
    
        public Task ReceiveReminder(string reminderName, TickStatus status)
        {
            // Ensure that the reminder task is running.
            if (_processQueueTask is null || _processQueueTask.IsCompleted)
            {
                if (_processQueueTask?.Exception is Exception exception)
                {
                    // Log that an error occurred.
                }
    
                _processQueueTask = DoLongRunningWork();
                _processQueueTask.Ignore();
            }
    
            return Task.CompletedTask;
        }
    
        public override async Task OnActivateAsync()
        {
            if (_reminder != null)
            {
                return;
            }
    
            _reminder = await RegisterOrUpdateReminder(
                "long-running-work",
                TimeSpan.FromMinutes(1),
                TimeSpan.FromMinutes(1)
            );
        }
    
        public override async Task OnDeactivateAsync()
        {
            _deactivating.Cancel(throwOnFirstException: false);
    
            Task processQueueTask = _processQueueTask;
            if (processQueueTask != null)
            {
                // Optionally add some max deactivation timeout here to stop waiting after (eg) 45 seconds
    
                await processQueueTask;
            }
        }
    
        public async Task StopAsync()
        {
            if (_reminder == null)
            {
                return;
            }
            await UnregisterReminder(_reminder);
            _reminder = null;
        }
    
        private async Task DoLongRunningWork()
        {
            // Log that we are starting the long-running work
            while (!_deactivating.IsCancellationRequested)
            {
                try
                {
                    // Do long-running work
                }
                catch (Exception exception)
                {
                    // Log exception. Potentially wait before retrying loop, since it seems like GetMessageAsync may have failed for us to end up here.
                }
            }
        }
    }