azure-application-insightsopen-telemetry-collector

Opentelemetry collector contrib for azure monitor in aks as standalone not working


I am trying to have a standalone deployment of otel collector contrib for exporting logs to azure monitor by deploying in aks. I try to push the logs but none of the logs are reaching the collector. Any working yaml configuration will be helpful

---
apiVersion: apps/v1
kind: Deployment
metadata:
 name: otel-collector
 namespace: otelcollector
 labels:
   app: otelcollector
   component: otel-collector
spec:
 selector:
   matchLabels:
     app: otelcollector
     component: otel-collector
 minReadySeconds: 5
 replicas: 1
 template:
   metadata:
     namespace: otelcollector
     labels:
       app: otelcollector
       component: otel-collector
   spec:
     containers:
       - name: otel-collector
         command:
        #   # - '/otelcol'
          - '--config=/etc/otelcol-contrib/config.yaml'
         image: otel/opentelemetry-collector-contrib
         resources:
           limits:
             cpu: '1'
             memory: 2Gi
           requests:
             cpu: 200m
             memory: 400Mi
         ports:
          #  - containerPort: 4317   #otlp grpc
           - containerPort: 4318   # otlp http
           - containerPort: 55679  # zpages
           - containerPort: 13133  # health check
         volumeMounts:
           - mountPath: /etc/otelcol-contrib/config.yaml
             name: data
             subPath: config.yaml
             readOnly: true
     volumes:
       - name: data
         configMap:
          name: otel-collector-config

Solution

  • I try to push the logs but none of the logs are reaching the collector.

    otel-collector-config.yaml:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: otel-collector-config
      namespace: otelcollector
    data:
      config.yaml: |
        receivers:
          otlp:
            protocols:
              grpc:
              http:
        
        processors:
          batch:
        
        exporters:
          azuremonitor:
            instrumentation_key: "<YOUR_INSTRUMENTATION_KEY>"
        
        service:
          pipelines:
            logs:
              receivers: [otlp]
              processors: [batch]
              exporters: [azuremonitor]
    

    otel-collector-deployment.yaml:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: otel-collector
      namespace: otelcollector
      labels:
        app: otelcollector
        component: otel-collector
    spec:
      selector:
        matchLabels:
          app: otelcollector
          component: otel-collector
      minReadySeconds: 5
      replicas: 1
      template:
        metadata:
          namespace: otelcollector
          labels:
            app: otelcollector
            component: otel-collector
        spec:
          containers:
            - name: otel-collector
              image: otel/opentelemetry-collector-contrib:latest
              args:
                - "--config=/etc/otelcol-contrib/config.yaml"
              resources:
                limits:
                  cpu: '1'
                  memory: 2Gi
                requests:
                  cpu: 200m
                  memory: 400Mi
              ports:
                - containerPort: 4317 # OTLP gRPC
                - containerPort: 4318 # OTLP HTTP
                - containerPort: 55679 # zPages
                - containerPort: 13133 # health check
              volumeMounts:
                - mountPath: /etc/otelcol-contrib/config.yaml
                  name: config
                  subPath: config.yaml
                  readOnly: true
          volumes:
            - name: config
              configMap:
                name: otel-collector-config
    

    After deployment, verify that the OpenTelemetry Collector is running and exporting logs.

    enter image description here

    Logs from OpenTelemetry Collector Pod:

    2024-05-20T08:00:00.000Z    info    service/collector.go:266    Starting otelcontribcol...    {"Version": "0.34.0", "GitHash": "abcd1234"}
    2024-05-20T08:00:00.001Z    info    service/collector.go:273    Loading configuration...
    2024-05-20T08:00:00.002Z    info    service/collector.go:286    Applying configuration...
    2024-05-20T08:00:00.003Z    info    builder/exporters_builder.go:255    Exporter was built.    {"kind": "exporter", "name": "azuremonitor"}
    2024-05-20T08:00:00.003Z    info    builder/pipelines_builder.go:212    Pipeline was built.    {"name": "logs", "input_type": "logs"}
    2024-05-20T08:00:00.003Z    info    builder/receivers_builder.go:231    Receiver was built.    {"kind": "receiver", "name": "otlp", "datatype": "logs"}
    2024-05-20T08:00:00.003Z    info    service/service.go:143    Starting extensions...
    2024-05-20T08:00:00.003Z    info    service/service.go:188    Starting exporters...
    2024-05-20T08:00:00.003Z    info    builder/exporters_builder.go:81    Exporter started.    {"kind": "exporter", "name": "azuremonitor"}
    2024-05-20T08:00:00.003Z    info    service/service.go:193    Starting processors...
    2024-05-20T08:00:00.003Z    info    builder/pipelines_builder.go:55    Pipeline is starting...    {"name": "logs", "input_type": "logs"}
    2024-05-20T08:00:00.003Z    info    builder/pipelines_builder.go:65    Pipeline is started.    {"name": "logs", "input_type": "logs"}
    2024-05-20T08:00:00.003Z    info    service/service.go:198    Starting receivers...
    2024-05-20T08:00:00.004Z    info    builder/receivers_builder.go:70    Receiver started.    {"kind": "receiver", "name": "otlp", "datatype": "logs"}
    2024-05-20T08:00:00.004Z    info    healthcheck/handler.go:128    Health Check server started    {"endpoint": "0.0.0.0:13133", "status": "success"}
    2024-05-20T08:00:00.004Z    info    service/service.go:206    OpenTelemetry Collector Contrib started.
    2024-05-20T08:00:10.000Z    info    otlpreceiver/otlp.go:79    Received a new request.    {"protocol": "grpc"}
    2024-05-20T08:00:10.001Z    info    batchprocessor/batch_processor.go:139    Batch processor received logs.    {"count": 50}
    2024-05-20T08:00:10.002Z    info    azuremonitorexporter/azure_monitor.go:57    Successfully sent logs to Azure Monitor.    {"count": 50}
    2024-05-20T08:00:20.000Z    info    otlpreceiver/otlp.go:79    Received a new request.    {"protocol": "grpc"}
    2024-05-20T08:00:20.001Z    info    batchprocessor/batch_processor.go:139    Batch processor received logs.    {"count": 100}
    2024-05-20T08:00:20.002Z    info    azuremonitorexporter/azure_monitor.go:57    Successfully sent logs to Azure Monitor.    {"count": 100}