azureazure-functionsappinsights

Azure Flex Consumption Apps not logging to App Insights

Our Flex Function Apps are no longer logging to App Insights since around 20th February - rather oddly it seems to have coincided with all our Logic Apps failing to log, which was due to the App Insights Instrumentation Key finally being deprecated in the config. Shifting all the Logic Apps to use Connection String instead solved the issue with Logic Apps, but our Flex Apps are still not working despite the correct configuration being present.

For clarity, we are using APPLICATIONINSIGHTS_CONNECTION_STRING in our Flex Consumption configuration and always have been. It was only some legacy Logic Apps using Instrumentation Key.

It seems a strange coincidence and I wonder whether Microsoft have changed something.

For clarity here is the relevant section of my Program.cs file

var host = new HostBuilder()
    .ConfigureFunctionsWorkerDefaults()
    .ConfigureServices(s =>
    {
        ...

        s.AddApplicationInsightsTelemetryWorkerService();
        s.ConfigureFunctionsApplicationInsights();        

        s.Configure<LoggerFilterOptions>(o => 
        {
            // The Application Insights SDK adds a default logging filter that instructs ILogger to capture only Warning and more severe logs. Application Insights requires an explicit override.
            // Log levels can also be configured using appsettings.json. For more information, see https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service#ilogger-logs
            LoggerFilterRule defaultRule = o.Rules.FirstOrDefault(r => r.ProviderName == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider")!;

            if (defaultRule is not null)
            {
                o.Rules.Remove(defaultRule);
            }
        });

        ...
    })
    .Build();

host.Run();

Relevant Nuget packages

<PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.22.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="2.0.0" />

Host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
        "samplingSettings": {
            "isEnabled": true,
            "excludedTypes": "Request"
        }
    }
  }
}

I am seeing startup information in the trace logs when the Flex App is first deployed, but there are no requests being logged. Again, this seems to have stopped randomly, and given Flex Apps are new and Microsoft are actively making changes I did wonder whether something has changed regarding the way Flex Consumption apps log to App Insights.

Solution

  • We solved our specific issue, here are the details in case it helps anyone else:

    Our Flex Apps are connected to a VNet while our Application Insights instance is not, and so I checked whether they could see the App Insights endpoints specified in the Connection String from within the VNet. On investigate I found the endpoints were not reachable. Specifically this URL from the Connection String could not be reached from within the Virtual Network: uksouth-1.in.applicationinsights.azure.com

    We had no specific Private DNS Zones configured for applicationinsights.azure.com. And so on doing an NSLOOKUP outside of the VNet I noticed the following aliases are used for that domain:

    enter image description here

    Interestingly we do have a Private DNS Zone for privatelink.monitor.azure.com, but we have always had that Private DNS Zone in place even when logs were working.

    On speaking with a colleague I discovered he too had a similar issue with Power BI, in that it suddenly stopped communicating with resources on the network. His specific issue was related with a separate Private DNS Zone, namely privatelink.pbidedicated.windows.net. The fix he applied was to enable the following option within the Virtual Network link attached to the Private DNS Zone:

    enter image description here

    I applied the same configuration, i.e. Enable fall-back to Internet, to our Private DNS Zone privatelink.monitor.azure.com, and all our logs started working again.

    Very recently we had a separate issue where privatelink.scm.azurewebsites.net stopped working in a very similar circumstance.

    TLDR: enabling "fallback to internet" seems to work if you don't have specific routes setup within the VNet to an Application Insights instance, and although I don't have any evidence, I'm suspicious that Microsoft have changed something behind the scenes regarding how Private DNS Zones work given the coincidence with other similar sudden failures.

    Posting this in the hope this helps someone.