azureazure-functionsazure-app-service-plansazure-storage-queues

Overwriting Autoscale Rules Of Azure App Service Plan By The Azure Function App?


I have an Azure function app that is under a Dedicated plan (Azure App Service Plan P1V2).

The Azure Function App has a Queue Trigger function for Azure Storage Queues.

I'm looking to approximate FIFO as close as possible, I realize true FIFO is not guaranteed with Azure Storage Queues, but the documentation also states that Azure Storage Queues and Queue triggers typically go follow first in first out. (see "Additional Information" https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted#additional-information)

my host.json is the following:

{
  "version": "2.0",
  "extensions": {
    "queues": {
      "batchSize": 1,
      "maxDequeueCount": 10,
      "newBatchThreshold": 0
    }
  }
}

I have the app setting of WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT = 1 which is supposed to limit the amount of VMs to 1 in an autoscale situation. (Side question: Is this app setting useless when the function app is under a dedicated plan? Is it only applicable for Consumption and Elastic Premium Plans? I ask because there is a similar sounding site config for Microsoft.Web/sites, arm template):

enter image description here

It seems that yes, my queue trigger is following one at a time execution, and I can't see "visible" evidence of more than 1 instance running (even during that high volume period).

However, the queue trigger still did not start in the order of queue insertion time. I have 3 messages being inserted, in a specific order, but the queue trigger processed message 3 before message 2 (again, high volume, there were other messages present)

I noticed my autoscale in the app service plan has some rules when CPU percentage gets too high, allowing up to a maximum of 10 instances, a minimum of 1, and a default of 1.

Can I somehow overwrite this autoscale at the app service plan level, by a setting at the Azure function level? Is this already being done by WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT = 1??


Solution

  • The setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT primarily applies to Consumption and Elastic Premium Plans, not to Dedicated plans like the one you're using (P1V2). In Dedicated plans, Azure Functions run on dedicated VM instances, so scaling behavior is managed differently, and this setting doesn't directly impact your function app.

    Autoscaling in Dedicated plans is controlled at the App Service Plan level, not within the function app itself. The autoscale rules you've configured at the App Service Plan level can override any attempts to limit scaling within your function app's configuration.

    You can create a Custom Autoscale rule to scale out your App based on a specific metric:-

    enter image description here

    You can modify autoscale settings in your App Service Plan to ensure only one instance runs at a time, but it may affect performance and scalability during peak loads. Consider implementing custom logic in your function app to enforce strict message ordering, like checking timestamps, regardless of the queue retrieval order.

    Custom logic:-

    using System;
    using Microsoft.Azure.WebJobs;
    using Microsoft.Extensions.Logging;
    
    namespace FunctionApp5
    {
        public class Function1
        {
            [FunctionName("Function1")]
            public void Run([ServiceBusTrigger("myqueue", Connection = "servicebusconn")] string myQueueItem, ILogger log)
            {
                try
                {
                    if (string.IsNullOrEmpty(myQueueItem))
                    {
                        log.LogError("Received empty or null message from the queue.");
                        return;
                    }
    
                    log.LogInformation($"Received message from the queue: {myQueueItem}");
    
                    var message = Newtonsoft.Json.JsonConvert.DeserializeObject<MyMessage>(myQueueItem);
    
                    if (message.Timestamp < previousMessageTimestamp)
                    {
                        
                        log.LogError($"Received out-of-order message at timestamp: {message.Timestamp}");
                    }
                    else
                    {
                        previousMessageTimestamp = message.Timestamp;
    
                        log.LogInformation($"Processing message received at timestamp: {message.Timestamp}");
                    }
                }
                catch (Exception ex)
                {
                    log.LogError($"An error occurred while processing the message: {ex.Message}");
                }
            }
    
            public class MyMessage
            {
                public DateTime Timestamp { get; set; }
                
            }
            private static DateTime previousMessageTimestamp = DateTime.MinValue;
        }
    }
    

    enter image description here

    enter image description here

    While Azure Storage Queues generally follow a FIFO (First In, First Out) pattern, the actual order of processing may vary due to factors like concurrency, message visibility timeouts, and retries. Even though your queue trigger is set to process one message at a time (with a batchSize of 1), concurrent execution of multiple instances of your function app, possibly due to autoscaling or other factors, can impact the order of message processing."