nservicebusnservicebus-sagas

Optimizing saga lookup


I have a proces that downloads files from a remote location in parallel threads. Each thread sends a message when a download is started, and a second one when the download is completed. Both messages have a download id property (guid) to correlate the two.

Next I have a saga that monitors these downloads. It is started by the DownloadStarted event, and uses a timeout to detect if the DownloadEnded event is received in time.

The problem I have is that the performance of the saga is not that great when a large amount of files are downloaded in a short time (1000 files in 1 minute). At certain times it takes more than half an hour for it to catch up.

I tried to speedup the saga lookup by providing an IFindSagas impementation. That didn't help much, as that caused RavenDB to create an auto index on the DownloadId in the saga data, but also caused the FindBy method to often return null because that index wasn't updated in time.

Is there any other way I could try to speed up the saga? I was thinking of using the DownloadId as saga id, as that is already a unique guid. The Id property of the saga data is settable, but the documentation specifically states that you should not set the id yourself...

Transport used: MSMQ Persistence used: RavenDB NServiceBus version: 5


Solution

  • I reworked the setup so that the saga is no longer needed. In the process that downloads the file, I now start a timer to send a notification in the background and when the download is completed I stop the timer. This works a lot faster and causes a lot less messages to be sent.

    using (var timer = new Timer(1000) {AutoReset = true})
    {
        var start = DateTime.Now;
        timer.Elapsed += (sender, e) =>
                          _bus.Send(new NotifyHangingDownload(correlationId,
                                                              file.Filename,
                                                              start,
                                                              TimeSpan.FromMilliseconds(1000)));
        timer.Start();
        client.Download(file, destination, false);
        timer.Stop();
    }