azure-webjobsazure-webjobssdkqueuetrigger

How to configure different batch sizes for different queues in "webjobs v3"?


There is some documentation on configuring different batch sizes for different queues using customQueueProcessor in webjobs v2 using .net framework. I would like to know how this is handled in webjobs v3?

       var builder = new HostBuilder()
            .UseEnvironment("Development")
            .ConfigureWebJobs(b =>
            {
                b.AddAzureStorageCoreServices();
                b.AddAzureStorage(a =>
                {
                    a.BatchSize = 1;
                });
            })

This batch size is applicable for all the QueueTriggers in the code. How to use custom values for different queues?


Solution

  • If you want to set BatchSize per queue, you can implement an IQueueProcessorFactory:

    public class CustomQueueProcessorFactory : IQueueProcessorFactory
    {
        public QueueProcessor Create(QueueProcessorFactoryContext context)
        {
            if (context.Queue.Name.Equals("fooqueue"))
            {
                // demonstrates how batch processing behavior can be customized
                // per queue (as opposed to the global settings that apply to ALL queues)
                context.BatchSize = 3;
                context.NewBatchThreshold = 4;
                ...
            }
    
            return new QueueProcessor(context);
        }
    }
    

    In this case - all queues will use your default BatchSize configuration, but queue trigger "fooqueue" will have BatchSize set to 3.

    Register CustomQueueProcessorFactory in your ConfigureServices method as so:

    builder.ConfigureServices((services) =>
    {
        services.AddSingleton<IQueueProcessorFactory, CustomQueueProcessorFactory>();
    });