How to optimize Application Insight costs for Azure Functions?

I have Azure Function(Python) and Azure Application Insight. Application Insight costs are huge, because it get 200GB data by month. I'm trying to track where is the issue.

I wonder Azure Function logs everything and is it possible to optimize. We would not need "info".

How to optimize costs in Application Insight for Azure Functions?

Application Insight:

logger = logging.getLogger(__name__)
    logger.info('Python HTTP trigger function received a request.')

    try:
        instrumentationKey = os.environ['APPINSIGHTS_INSTRUMENTATIONKEY']
        logger.addHandler(AzureLogHandler(
            connection_string=f'InstrumentationKey={instrumentationKey}')
        )
    except Exception as e:
        exception_text = f"{e}"
        logging.error(f"Could not add Application Insights logging: {exception_text}")

Logging usage:

 logging.error(f"EXCEPTION: {exception_text}")
 logging.info(f" Calling Activity Function")

I wonder Azure Function logs everything and is it possible to optimize. We would not need "info".

1. Disable unneeded modules: Edit the ApplicationInsights.config to turn off collection modules which are not required.

• Application Insights .NET SDK consists of a number of NuGet Packages that contains an API for sending telemetry and it also consists of telemetry modules and initializers for automatically tracking from your application and its context. You can enable or disable Telemetry Modules and initializers, and set parameters for some of them.

• There's a node in the configuration file for each module. To disable a module, delete the node or comment it out.

2. Disable telemetry dynamically: To disable telemetry conditionally and dynamically anywhere in the code, set the DisableTelemetry flag on it using TelemetryConfiguration instance.

public void ConfigureServices(IServiceCollection services)
    {
        services.AddApplicationInsightsTelemetry();
    }

public void Configure(IApplicationBuilder app, IHostingEnvironment env, TelemetryConfiguration configuration)
{
    configuration.DisableTelemetry = true;
    ...
}

This code sample prevents the sending of telemetry to Application Insights but not the automatic collection modules from colleting telemetry and in order to remove auto collection modules also, please refer this Microsoft Documentation.

3. Customize Logs Collection:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    },
    "ApplicationInsights": {
      "LogLevel": {
        "Default": "Information"
      }
    }
  }
}

The following above configuration allows Application Insights to capture all Information logs and severe warning logs. To change this behavior, explicitly override the logging configuration for the provider ApplicationInsights as shown below:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  }
}

There are few more techniques to manage the data volume used for telemetry data optimizing also like:

• Sampling: Sampling is the primary tool used to tune the amount of data you send. Sampling in Application Insights
• Daily Cap: The maximum cap in Application Insights is 1000 GB/day unless you request a higher maximum for a high-traffic application.
• If you have workspace-based Application Insights resource, we recommend using the workspace's daily cap to limit ingestion and costs instead of the cap in Application Insights.
• Pre-aggregate metrics: If you put calls to TrackMetric in your app, you can reduce traffic by using the overload that accepts your calculation of the average and standard deviation of a batch of measurements. Or, you can use a pre-aggregating package.

Please check these references for more information:

1. Resolve if logs shows twice in Application Insights
2. Optimizing logging costs of Azure Functions
3. Configuring or removing the necessary Telemetry Initializers