I am trying to have a standalone deployment of otel collector contrib for exporting logs to azure monitor by deploying in aks. I try to push the logs but none of the logs are reaching the collector. Any working yaml configuration will be helpful
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: otel-collector
namespace: otelcollector
labels:
app: otelcollector
component: otel-collector
spec:
selector:
matchLabels:
app: otelcollector
component: otel-collector
minReadySeconds: 5
replicas: 1
template:
metadata:
namespace: otelcollector
labels:
app: otelcollector
component: otel-collector
spec:
containers:
- name: otel-collector
command:
# # - '/otelcol'
- '--config=/etc/otelcol-contrib/config.yaml'
image: otel/opentelemetry-collector-contrib
resources:
limits:
cpu: '1'
memory: 2Gi
requests:
cpu: 200m
memory: 400Mi
ports:
# - containerPort: 4317 #otlp grpc
- containerPort: 4318 # otlp http
- containerPort: 55679 # zpages
- containerPort: 13133 # health check
volumeMounts:
- mountPath: /etc/otelcol-contrib/config.yaml
name: data
subPath: config.yaml
readOnly: true
volumes:
- name: data
configMap:
name: otel-collector-config
I try to push the logs but none of the logs are reaching the collector.
otel-collector-config.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: otel-collector-config
namespace: otelcollector
data:
config.yaml: |
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
exporters:
azuremonitor:
instrumentation_key: "<YOUR_INSTRUMENTATION_KEY>"
service:
pipelines:
logs:
receivers: [otlp]
processors: [batch]
exporters: [azuremonitor]
config.yaml
from the ConfigMap and check that the Collector is started with the appropriate configuration.otel-collector-deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: otel-collector
namespace: otelcollector
labels:
app: otelcollector
component: otel-collector
spec:
selector:
matchLabels:
app: otelcollector
component: otel-collector
minReadySeconds: 5
replicas: 1
template:
metadata:
namespace: otelcollector
labels:
app: otelcollector
component: otel-collector
spec:
containers:
- name: otel-collector
image: otel/opentelemetry-collector-contrib:latest
args:
- "--config=/etc/otelcol-contrib/config.yaml"
resources:
limits:
cpu: '1'
memory: 2Gi
requests:
cpu: 200m
memory: 400Mi
ports:
- containerPort: 4317 # OTLP gRPC
- containerPort: 4318 # OTLP HTTP
- containerPort: 55679 # zPages
- containerPort: 13133 # health check
volumeMounts:
- mountPath: /etc/otelcol-contrib/config.yaml
name: config
subPath: config.yaml
readOnly: true
volumes:
- name: config
configMap:
name: otel-collector-config
After deployment, verify that the OpenTelemetry Collector is running and exporting logs.
Logs from OpenTelemetry Collector
Pod:
2024-05-20T08:00:00.000Z info service/collector.go:266 Starting otelcontribcol... {"Version": "0.34.0", "GitHash": "abcd1234"}
2024-05-20T08:00:00.001Z info service/collector.go:273 Loading configuration...
2024-05-20T08:00:00.002Z info service/collector.go:286 Applying configuration...
2024-05-20T08:00:00.003Z info builder/exporters_builder.go:255 Exporter was built. {"kind": "exporter", "name": "azuremonitor"}
2024-05-20T08:00:00.003Z info builder/pipelines_builder.go:212 Pipeline was built. {"name": "logs", "input_type": "logs"}
2024-05-20T08:00:00.003Z info builder/receivers_builder.go:231 Receiver was built. {"kind": "receiver", "name": "otlp", "datatype": "logs"}
2024-05-20T08:00:00.003Z info service/service.go:143 Starting extensions...
2024-05-20T08:00:00.003Z info service/service.go:188 Starting exporters...
2024-05-20T08:00:00.003Z info builder/exporters_builder.go:81 Exporter started. {"kind": "exporter", "name": "azuremonitor"}
2024-05-20T08:00:00.003Z info service/service.go:193 Starting processors...
2024-05-20T08:00:00.003Z info builder/pipelines_builder.go:55 Pipeline is starting... {"name": "logs", "input_type": "logs"}
2024-05-20T08:00:00.003Z info builder/pipelines_builder.go:65 Pipeline is started. {"name": "logs", "input_type": "logs"}
2024-05-20T08:00:00.003Z info service/service.go:198 Starting receivers...
2024-05-20T08:00:00.004Z info builder/receivers_builder.go:70 Receiver started. {"kind": "receiver", "name": "otlp", "datatype": "logs"}
2024-05-20T08:00:00.004Z info healthcheck/handler.go:128 Health Check server started {"endpoint": "0.0.0.0:13133", "status": "success"}
2024-05-20T08:00:00.004Z info service/service.go:206 OpenTelemetry Collector Contrib started.
2024-05-20T08:00:10.000Z info otlpreceiver/otlp.go:79 Received a new request. {"protocol": "grpc"}
2024-05-20T08:00:10.001Z info batchprocessor/batch_processor.go:139 Batch processor received logs. {"count": 50}
2024-05-20T08:00:10.002Z info azuremonitorexporter/azure_monitor.go:57 Successfully sent logs to Azure Monitor. {"count": 50}
2024-05-20T08:00:20.000Z info otlpreceiver/otlp.go:79 Received a new request. {"protocol": "grpc"}
2024-05-20T08:00:20.001Z info batchprocessor/batch_processor.go:139 Batch processor received logs. {"count": 100}
2024-05-20T08:00:20.002Z info azuremonitorexporter/azure_monitor.go:57 Successfully sent logs to Azure Monitor. {"count": 100}