Open telemetry pod fails on AKS with failed to get config
I am applying the following yaml manifest to the AKS cluster
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
name: default-otel-collector
annotations:
# We want to create this after the CRD's are created. Argo starts at zero
argocd.argoproj.io/sync-wave: "1"
spec:
mode: deployment
autoscaler:
maxReplicas: 18
minReplicas: 2
targetCPUUtilization: 80
targetMemoryUtilization: 80
resources:
requests:
cpu: 100m
memory: 1Gi
limits:
cpu: 100m
memory: 1Gi
# Only needed for daemonsets which aren't used here
#serviceAccount: default-otel-collector
tolerations:
- operator: Exists
#hostNetwork: true
# Environment variables here can be passed to the config
env:
- name: HONEYCOMB_ENDPOINT
value: "api.honeycomb.io:443"
- name: REGION
value: "REPLACE_ME"
- name: LOG_LEVEL
value: "INFO"
# When honeycomb is ready, uncomment this
# The collector will fail if the secret isn't there
- name: HONEYCOMB_API_KEY
valueFrom:
secretKeyRef:
name: honeycomb-auth
key: api-key
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
#ports:
config: |
receivers:
jaeger:
protocols:
grpc:
thrift_binary:
thrift_compact:
thrift_http:
otlp:
protocols:
grpc:
http:
processors:
batch:
memory_limiter:
check_interval: 1s
limit_percentage: 90
spike_limit_percentage: 10
#k8sattributes:
# filter:
# node_from_env_var: KUBE_NODE_NAME
attributes/eks:
actions:
- key: collector_host
value: "$KUBE_NODE_NAME"
action: upsert
- key: region
value: "$REGION"
action: upsert
transform/refreshbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "mentos:REFRESH_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "mentos:REFRESH_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "mentos:REFRESH_IDENTITY receive"
transform/refreshbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "mentos:REFRESH_IDENTITY receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "mentos:REFRESH_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "mentos:REFRESH_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "mentos:REFRESH_IDENTITY receive"
- set(parent_span_id.string, "0000000000000000") where name == "mentos:REFRESH_IDENTITY receive"
transform/aggregationbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
transform/aggregationbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
- set(parent_span_id.string, "0000000000000000") where name == "qpoc:ACCOUNT_RESOURCE_AGGREGATION receive"
transform/deletebreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
transform/deletebreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
- set(parent_span_id.string, "0000000000000000") where name == "qpoc:IDENTITY_ACCOUNT_DELETE receive"
transform/wpsbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "wps:REQUEST receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "wps:REQUEST receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "wps:REQUEST receive"
transform/wpsbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "wps:REQUEST receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "wps:REQUEST receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "wps:REQUEST receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "wps:REQUEST receive"
- set(parent_span_id.string, "0000000000000000") where name == "wps:REQUEST receive"
transform/streamingaggbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "StreamingAccountResourceAggregation.handleMessageInternal"
transform/streamingaggbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(attributes["sre.test.originRoot"], "yes") where name == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "StreamingAccountResourceAggregation.handleMessageInternal"
- set(parent_span_id.string, "0000000000000000") where name == "StreamingAccountResourceAggregation.handleMessageInternal"
transform/promoAttrbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
transform/promoAttrbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
- set(parent_span_id.string, "0000000000000000") where name == "mentos:IDENTITY_ATTRIBUTE_PROMOTION_REQUEST receive"
transform/identChangebreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "identity:IDENTITY_CHANGED receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "identity:IDENTITY_CHANGED receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "identity:IDENTITY_CHANGED receive"
transform/identChangebreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "identity:IDENTITY_CHANGED receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "identity:IDENTITY_CHANGED receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "identity:IDENTITY_CHANGED receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "identity:IDENTITY_CHANGED receive"
- set(parent_span_id.string, "0000000000000000") where name == "identity:IDENTITY_CHANGED receive"
transform/batchSourcebreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
transform/batchSourcebreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
- set(parent_span_id.string, "0000000000000000") where name == "BATCHED_SOURCE_ACCOUNTS_RESPONSE receive"
transform/qpocRefreshbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "qpoc:REFRESH_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:REFRESH_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:REFRESH_IDENTITY receive"
transform/qpocRefreshbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "qpoc:REFRESH_IDENTITY receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "qpoc:REFRESH_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:REFRESH_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:REFRESH_IDENTITY receive"
- set(parent_span_id.string, "0000000000000000") where name == "qpoc:REFRESH_IDENTITY receive"
transform/qpocSyncbreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "qpoc:SYNCHRONIZE_IDENTITY receive"
transform/qpocSyncbreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(attributes["sre.test.originRoot"], "yes") where name == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "qpoc:SYNCHRONIZE_IDENTITY receive"
- set(parent_span_id.string, "0000000000000000") where name == "qpoc:SYNCHRONIZE_IDENTITY receive"
transform/mentosAttrPrombreakChild:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where attributes["sp.parent_name"] == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(attributes["sre.test.id"], Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(trace_id.string, Concat(["1000000000000000", parent_span_id.string], "")) where attributes["sp.parent_name"] == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
transform/mentosAttrPrombreakParent:
error_mode: ignore
trace_statements:
- context: span
statements:
- set(attributes["sre.test.originTrace"], trace_id.string) where name == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(attributes["sre.test.originRoot"], "yes") where name == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(attributes["sre.test.id"], Concat(["1000000000000000", span_id.string], "")) where name == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(trace_id.string, Concat(["1000000000000000", span_id.string], "")) where name == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
- set(parent_span_id.string, "0000000000000000") where name == "IdentityAttributePromotionMessageHandler.doInlineRefreshWithLock"
filter/dropaudit1:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.iris.event_type'
value: AUDIT_WHITELISTED
filter/dropmentos2:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: IDENTITY_ATTRIBUTE_PROMOTION_REQUEST
filter/dropqpoc1:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: STREAMING_ACCOUNT_RESOURCE_AGGREGATION
filter/dropqpoc2:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: STREAMING_IDENTITY_MANAGER_CORRELATION
filter/dropqpoc3:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: IDENTITY_ACCOUNT_DELETE
filter/dropqpoc4:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: ACCOUNT_RESOURCE_AGGREGATION
filter/drophecate1:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: 'STREAMING_ACCOUNT_RESOURCE_AGGREGATION'
filter/dropcreatecert:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: CREATE_CERTIFICATION
filter/dropaudit2:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.messaging.payload_type'
value: AUDIT_EVENT
filter/dropaudit3:
spans:
exclude:
match_type: strict
attributes:
- key: 'name'
value: 'CloudProvisioningLibrary.createAuditEvent'
filter/dropaudit4:
spans:
exclude:
match_type: strict
attributes:
- key: 'name'
value: 'CloudProvisioningLibrary.auditIdentityRequestItems'
filter/dropaudit5:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.parent_name'
value: 'CloudProvisioningLibrary.auditIdentityRequestItems'
filter/dropaudit6:
spans:
exclude:
match_type: strict
attributes:
- key: 'sp.parent_name'
value: 'CloudProvisioningLibrary.generateAuditsFromItems'
filter/droptoofast:
spans:
exclude:
match_type: regexp
attributes:
- key: 'duration_ms'
value: ^0\.0(1|0)[0-9]*
exporters:
otlphttp:
endpoint: "$HONEYCOMB_ENDPOINT"
headers:
"x-honeycomb-team": "$HONEYCOMB_API_KEY"
extensions:
health_check:
pprof:
zpages:
service:
extensions: [health_check, pprof, zpages]
pipelines:
traces:
receivers: [jaeger,otlp]
processors:
- batch
- memory_limiter
- attributes/eks
- transform/refreshbreakChild
- transform/refreshbreakParent
- transform/aggregationbreakChild
- transform/aggregationbreakParent
- transform/streamingaggbreakChild
- transform/streamingaggbreakParent
- transform/identChangebreakChild
- transform/identChangebreakParent
- transform/promoAttrbreakChild
- transform/promoAttrbreakParent
- transform/wpsbreakChild
- transform/wpsbreakParent
- transform/deletebreakChild
- transform/deletebreakParent
- transform/batchSourcebreakChild
- transform/batchSourcebreakParent
- transform/qpocRefreshbreakChild
- transform/qpocRefreshbreakParent
- transform/qpocSyncbreakChild
- transform/qpocSyncbreakParent
- transform/mentosAttrPrombreakChild
- transform/mentosAttrPrombreakParent
- filter/dropaudit1
- filter/dropaudit2
- filter/dropaudit3
- filter/dropaudit4
- filter/dropaudit5
- filter/dropaudit6
- filter/dropmentos2
- filter/dropqpoc1
- filter/dropqpoc2
- filter/dropqpoc3
- filter/dropqpoc4
- filter/dropcreatecert
- filter/drophecate1
- filter/droptoofast
exporters: [otlphttp]
telemetry:
logs:
level: "$LOG_LEVEL"
I can see in both default-otel-collector-collector-1 and default-otel-collector-collector-2 pods the following event - Back-off restarting failed container otc-container in pod default-otel-collector-collector-1/2_monitoring(11c0586a-e937-420e-8551-a0d9fbcb73d6)
When I look into pods log I can see the following message
Error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:* error decoding 'processors': unknown type: "transform" for id: "transform/deletebreakChild" (valid values: [resource span probabilistic_sampler filter batch memory_limiter attributes])2023/11/21 21:55:07 collector server run finished with error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:* error decoding 'processors': unknown type: "transform" for id: "transform/deletebreakChild" (valid values: [resource span probabilistic_sampler filter batch memory_limiter attributes])
Solution
In order to successfully deploy Open Telemetry Collector on AKS, whatever may be the deployment mode (using helm, etc.), you’ll probably need a configuration file for your collector, you will need a deployment file, a service file and some app to capture the logs. I am creating the same as below-
$ vim configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: collector-config
data:
collector.yaml: |
receivers:
otlp:
protocols:
grpc:
processors:
exporters:
logging:
service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
---
apiVersion: v1
kind: ConfigMap
metadata:
name: agent-config
data:
agent.yaml: |
receivers:
otlp:
protocols:
grpc:
processors:
exporters:
otlp:
endpoint: "opentelemetrycollector.default.svc.cluster.local:4317"
insecure: true
service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [otlp]
$ vim opentelemetry.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: opentelemetrycollector
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: opentelemetrycollector
template:
metadata:
labels:
app.kubernetes.io/name: opentelemetrycollector
spec:
containers:
- name: otelcol
args:
- --config=/conf/collector.yaml
image: otel/opentelemetry-collector:0.18.0
volumeMounts:
- mountPath: /conf
name: collector-config
volumes:
- configMap:
items:
- key: collector.yaml
path: collector.yaml
name: collector-config
name: collector-config
$ vim service.yaml
apiVersion: v1
kind: Service
metadata:
name: opentelemetrycollector
spec:
ports:
- name: grpc-otlp
port: 4317
protocol: TCP
targetPort: 4317
selector:
app.kubernetes.io/name: opentelemetrycollector
type: ClusterIP
Output:
$ kubectl apply -f configmap.yaml
configmap/collector-config created
configmap/agent-config created
$ kubectl apply -f opentelemetry.yaml
deployment.apps/opentelemetrycollector created
$ kubectl apply -f service.yaml
service/opentelemetrycollector created
$ vim business-app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: myapp
template:
metadata:
labels:
app.kubernetes.io/name: myapp
spec:
containers:
- name: myapp
image: quay.io/jpkroehling/generate-span-java:0.1.0
- name: agent
image: otel/opentelemetry-collector:0.18.0
args:
- --config=/conf/agent.yaml
volumeMounts:
- mountPath: /conf
name: agent-config
volumes:
- configMap:
items:
- key: agent.yaml
path: agent.yaml
name: agent-config
name: agent-config
Output:
$ kubectl apply -f business-app.yaml
deployment.apps/myapp created
We now have a collector instance up and running, exposed the OTLP port that we declared in our configuration and there is a service definition that satisfies the requirement. Time to test it. Yes, I can see both my collector and 'myapp' application pods are up and able to collect the logs.
Reference Documents:
https://opentelemetry.io/docs/collector/transforming-telemetry/
https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/README.md
https://opentelemetry.io/docs/collector/configuration/#processors
- How do I get a certificate (public and private key) into a windows container in AKS?
- What is preventing AKS from pulling images from ACR?
- Not possible to use containers in Azure Pipelines self-hosted agents deployed as Azure K8s pods
- Shell (ssh) into Azure AKS (Kubernetes) cluster worker node
- How to enable Calico network policy on the existing AKS cluster
- Azure Kubernetes Service (AKS) uses Application Gateway Ingress Controller (AGIC) : How to implement HSTS header in ASP.Net Core 6.0?
- kubectl - How to restart a deployment (or all deployment)
- Can't access public IP of AKS ingress
- Azure CLI aks install cli permission denied and sudo does not work
- How do scale down statefulset when it's spread between nodepools?
- Can I access a container in a pod in an AKS cluster as root?
- Unable to push image to ACR from terraform - could not get auth config (the credentialhelper did not work or was not found):
- Create kubernetes secret for docker registry - Terraform
- No Frameworks Were Found When Deploying Background Service in AKS
- How to execute kubectl commands in HCP?
- How to connect HCP to authenticate to AKS cluster for deploying Helm and K8 resources?
- AKS with application gateway - always 502
- How to set pod system time in the Rancher Edit Yaml setting?
- terraform helm release resource fails but Helm command works
- How to restrict access to Keycloak admin console to a specific IP/IP range?
- Access Forbidden while accessing log in airflow with CeleryExecutor
- AGIC associating rules with defaultaddresspool instead of the newly created pool
- startup exception for aspnet core worker docker container attempting to set secret from env variable in Kubernetes
- AKS Network Policy Manager vs Azure Cilium Network Policies
- AKS Keda: Pods are removed during execution
- Ingress controller does not allow snippets
- AKS - Error from server (Forbidden): User cannot create resource "namespaces" in API group "" at the cluster scope
- Changing Prometheus job label in scraper for cAdvisor breaks Grafana dashboards
- How to disable port probes for AKS LoadBalancer?
- Airflow ERROR - Error authorizing OAuth access token: Invalid JSON Web Key Set. [The request to sign in was denied.]