djangoappinsightsopencensus

DJANGO OPENCENSUS url field in request data too long

I am having a similar issue to this question since adding App Insights to my application. It may be related to this other question also, but neither of them are directly related to App Insights and neither have solutions.

This is the error from the django-tasks.log

Data drop 400: 100: Field 'url' on type 'RequestData' is too long. Expected: 2048 characters, Actual: 3701 {'iKey': <uuid>, 'tags': {'ai.cloud.role': 'manage.py', 'ai.cloud.roleInstance': <instance>, 'ai.device.id': <device>, 'ai.device.locale': 'en_US', 'ai.device.osVersion': '#1 SMP Tue Aug 25 17:23:54 UTC 2020', 'ai.device.type': 'Other', 'ai.internal.sdkVersion': 'py3.6.12:oc0.7.11:ext1.0.4', 'ai.operation.id': 'fcbe18bf6ca9036aa4546af171f3e877', 'ai.operation.name': 'GET /<my_url>/'}, 'time': '2020-12-15T17:58:36.498868Z', 'name': 'Microsoft.ApplicationInsights.Request', 'data': {'baseData': {'id': '116a0658b513bdb9', 'duration': '0.00:00:00.096', 'responseCode': '200', 'success': True, 'properties': {'request.name': 'GET /<my_url>/', 'request.url': 'https://<my host>/<my_url>/?<my very long query string>', 'django.user.id': '90', 'django.user.name': '100044505'}, 'ver': 2, 'name': 'GET /<my_url>/', 'url': 'https://<my host>/<my_url>/?<my very long query string>', 'source': None, 'measurements': None}, 'baseType': 'RequestData'}, 'ver': 1, 'sampleRate': None, 'seq': None, 'flags': None}.

We also see this repeating in the logs.

Queue is full. Dropping telemetry.
Queue is full. Dropping telemetry.
Queue is full. Dropping telemetry.
Queue is full. Dropping telemetry.
Queue is full. Dropping telemetry.
Queue is full. Dropping telemetry.

I could rewrite the app to use shorter queries, but that seems like the wrong answer. Is there a way to configure django to support long URLs.

Solution

  • The buffer cannot be changed, but you can limit the size of the URL with a filter. In order to customize the trace exported, it has to be instantiated separately.

    def shorten_url(envelope):
if 25 < len(envelope.data.baseData.url):
    envelope.data.baseData["url"] = envelope.data.baseData.url[:25]+"..." 
return True

from opencensus.ext.azure.trace_exporter import AzureExporter
exporter = AzureExporter(service_name='mysite')
exporter.add_telemetry_processor(shorten_url)

OPENCENSUS = {
    'TRACE': {
         'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=1)',
         'EXPORTER': exporter
     #Assumes Environmental Variable 'APPINSIGHTS_INSTRUMENTATIONKEY'
    }
}

    Complete working example: https://github.com/Gamecock/Django-appinsights-example