open-telemetryjaegeropentracingdistributed-tracingbadgerdb

How to configure Badger DB for Jaeger all-in-one deployment using config file on Windows server


I am trying to setup jaeger-all-in-one on one windows server [without Docker] with Badger DB for persistent storage to test it.

Used the following config file to run the Jaeger by command jaeger-all-in-one.exe" --config-file="C:\Users\Administrator\Downloads\jaeger-config.yaml"

# Global options
sampler:
  type: const
  param: 1  # 1 means to sample all traces

log-level: debug
storage:
  type: badger  # Use Badger as storage backend
  options:
    directory-key: C:\Program Files\Jaeger\Badger\Data  # Path to store Badger data
    directory-value: C:\Program Files\Jaeger\Badger\Data # Path to store Badger value data
    max-cache-size: 10MB  # Size of Badger cache
    gc-interval: 10m  # Badger garbage collection interval
    key-timestamp: true  # Enable key timestamps for better querying
    ephemeral: false  # Set ephemeral to false for persistent storage
    span-retention: "7d"  # Retain trace data for 7 days
    dependency-retention: "7d"  # Retain dependency data for 7 days

# HTTP server settings
http:
  port: 16686  # Port for the Jaeger query service, adjust as necessary

# gRPC server settings
grpc:
  port: 4317  # Port for the Jaeger collector service, adjust as necessary

But the Jaeger is not using Badger storage instead it uses memory storage "msg":"Memory storage initialized". Reference attached entire debug logs,

2024/04/15 04:57:46 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
2024/04/15 04:57:46 application version: git-commit=ecbae67ea32f189df1ddb4ec2da46d5fcd328b03, git-version=v1.56.0, build-date=2024-04-03T19:57:40Z
{"level":"info","ts":1713157066.1173787,"caller":"flags/service.go:110","msg":"Mounting metrics handler on admin server","route":"/metrics"}
{"level":"info","ts":1713157066.1173787,"caller":"flags/service.go:116","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
{"level":"info","ts":1713157066.1173787,"caller":"flags/admin.go:130","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","ts":1713157066.1173787,"caller":"flags/admin.go:144","msg":"Starting admin HTTP server","http-addr":":14269"}
{"level":"info","ts":1713157066.1173787,"caller":"flags/admin.go:122","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"}
{"level":"info","ts":1713157066.1173787,"caller":"grpc@v1.62.1/clientconn.go:429","msg":"[core][Channel #1] Channel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1173787,"caller":"grpc@v1.62.1/clientconn.go:1724","msg":"[core][Channel #1] original dial target is: \"localhost:4317\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1173787,"caller":"grpc@v1.62.1/clientconn.go:1731","msg":"[core][Channel #1] parsed dial target is: resolver.Target{URL:url.URL{Scheme:\"localhost\", Opaque:\"4317\", User:(*url.Userinfo)(nil), Host:\"\", Path:\"\", RawPath:\"\", OmitHost:false, ForceQuery:false, RawQuery:\"\", Fragment:\"\", RawFragment:\"\"}}","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.123022,"caller":"grpc@v1.62.1/clientconn.go:1745","msg":"[core][Channel #1] fallback to scheme \"passthrough\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1396246,"caller":"grpc@v1.62.1/clientconn.go:1753","msg":"[core][Channel #1] parsed dial target is: passthrough:///localhost:4317","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1403294,"caller":"grpc@v1.62.1/clientconn.go:1876","msg":"[core][Channel #1] Channel authority set to \"localhost:4317\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1415153,"caller":"grpc@v1.62.1/resolver_wrapper.go:197","msg":"[core][Channel #1] Resolver state updated: {\n  \"Addresses\": [\n    {\n      \"Addr\": \"localhost:4317\",\n      \"ServerName\": \"\",\n      \"Attributes\": null,\n      \"BalancerAttributes\": null,\n      \"Metadata\": null\n    }\n  ],\n  \"Endpoints\": [\n    {\n      \"Addresses\": [\n        {\n          \"Addr\": \"localhost:4317\",\n          \"ServerName\": \"\",\n          \"Attributes\": null,\n          \"BalancerAttributes\": null,\n          \"Metadata\": null\n        }\n      ],\n      \"Attributes\": null\n    }\n  ],\n  \"ServiceConfig\": null,\n  \"Attributes\": null\n} (resolver returned new addresses)","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1428385,"caller":"grpc@v1.62.1/balancer_wrapper.go:161","msg":"[core][Channel #1] Channel switches to new LB policy \"pick_first\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1434665,"caller":"grpc@v1.62.1/balancer_wrapper.go:213","msg":"[core][Channel #1 SubChannel #2] Subchannel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1447263,"caller":"grpc@v1.62.1/clientconn.go:532","msg":"[core][Channel #1] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.145355,"caller":"grpc@v1.62.1/clientconn.go:335","msg":"[core][Channel #1] Channel exiting idle mode","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.145355,"caller":"grpc@v1.62.1/clientconn.go:1223","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1460228,"caller":"memory/factory.go:85","msg":"Memory storage initialized","configuration":{"MaxTraces":0}}
{"level":"info","ts":1713157066.146683,"caller":"grpc@v1.62.1/clientconn.go:1338","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"localhost:4317\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1511598,"caller":"static/strategy_store.go:211","msg":"No sampling strategies provided or URL is unavailable, using defaults"}
{"level":"warn","ts":1713157066.1633234,"caller":"grpc@v1.62.1/clientconn.go:1400","msg":"[core][Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: \"localhost:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp [::1]:4317: connectex: No connection could be made because the target machine actively refused it.\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1665883,"caller":"grpc@v1.62.1/clientconn.go:1225","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to TRANSIENT_FAILURE, last error: connection error: desc = \"transport: Error while dialing: dial tcp [::1]:4317: connectex: No connection could be made because the target machine actively refused it.\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1691306,"caller":"grpc@v1.62.1/clientconn.go:532","msg":"[core][Channel #1] Channel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1691306,"caller":"grpc@v1.62.1/server.go:679","msg":"[core][Server #3] Server created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1755738,"caller":"server/grpc.go:104","msg":"Starting jaeger-collector gRPC server","grpc.host-port":"[::]:14250"}
{"level":"info","ts":1713157066.176147,"caller":"server/http.go:56","msg":"Starting jaeger-collector HTTP server","http host-port":":14268"}
{"level":"info","ts":1713157066.1765969,"caller":"grpc@v1.62.1/server.go:879","msg":"[core][Server #3 ListenSocket #4] ListenSocket created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.177802,"caller":"app/collector.go:146","msg":"Not listening for Zipkin HTTP traffic, port not configured"}
{"level":"info","ts":1713157066.1793082,"caller":"handler/otlp_receiver.go:77","msg":"OTLP receiver status change","status":"StatusStarting"}
{"level":"warn","ts":1713157066.1793082,"caller":"internal@v0.97.0/warning.go:42","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. Enable the feature gate to change the default and remove this warning.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks","feature gate ID":"component.UseLocalHostAsDefaultHost"}
{"level":"info","ts":1713157066.1805592,"caller":"grpc@v1.62.1/server.go:679","msg":"[core][Server #5] Server created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1812196,"caller":"otlpreceiver@v0.97.0/otlp.go:102","msg":"Starting GRPC server","endpoint":"0.0.0.0:4317"}
{"level":"warn","ts":1713157066.1818514,"caller":"internal@v0.97.0/warning.go:42","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. Enable the feature gate to change the default and remove this warning.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks","feature gate ID":"component.UseLocalHostAsDefaultHost"}
{"level":"info","ts":1713157066.1831172,"caller":"otlpreceiver@v0.97.0/otlp.go:152","msg":"Starting HTTP server","endpoint":"0.0.0.0:4318"}
{"level":"info","ts":1713157066.1818514,"caller":"grpc@v1.62.1/server.go:879","msg":"[core][Server #5 ListenSocket #6] ListenSocket created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1842952,"caller":"grpc/builder.go:74","msg":"Agent requested insecure grpc connection to collector(s)"}
{"level":"info","ts":1713157066.1858487,"caller":"grpc@v1.62.1/clientconn.go:429","msg":"[core][Channel #7] Channel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1885502,"caller":"grpc@v1.62.1/clientconn.go:1724","msg":"[core][Channel #7] original dial target is: \"localhost:14250\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1943777,"caller":"grpc@v1.62.1/clientconn.go:1731","msg":"[core][Channel #7] parsed dial target is: resolver.Target{URL:url.URL{Scheme:\"localhost\", Opaque:\"14250\", User:(*url.Userinfo)(nil), Host:\"\", Path:\"\", RawPath:\"\", OmitHost:false, ForceQuery:false, RawQuery:\"\", Fragment:\"\", RawFragment:\"\"}}","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1949522,"caller":"grpc@v1.62.1/clientconn.go:1745","msg":"[core][Channel #7] fallback to scheme \"passthrough\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.19587,"caller":"grpc@v1.62.1/clientconn.go:1753","msg":"[core][Channel #7] parsed dial target is: passthrough:///localhost:14250","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1967628,"caller":"grpc@v1.62.1/clientconn.go:1876","msg":"[core][Channel #7] Channel authority set to \"localhost:14250\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1979344,"caller":"grpc@v1.62.1/resolver_wrapper.go:197","msg":"[core][Channel #7] Resolver state updated: {\n  \"Addresses\": [\n    {\n      \"Addr\": \"localhost:14250\",\n      \"ServerName\": \"\",\n      \"Attributes\": null,\n      \"BalancerAttributes\": null,\n      \"Metadata\": null\n    }\n  ],\n  \"Endpoints\": [\n    {\n      \"Addresses\": [\n        {\n          \"Addr\": \"localhost:14250\",\n          \"ServerName\": \"\",\n          \"Attributes\": null,\n          \"BalancerAttributes\": null,\n          \"Metadata\": null\n        }\n      ],\n      \"Attributes\": null\n    }\n  ],\n  \"ServiceConfig\": null,\n  \"Attributes\": null\n} (resolver returned new addresses)","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.1995714,"caller":"grpc@v1.62.1/balancer_wrapper.go:161","msg":"[core][Channel #7] Channel switches to new LB policy \"round_robin\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2002034,"caller":"grpc@v1.62.1/balancer_wrapper.go:213","msg":"[core][Channel #7 SubChannel #8] Subchannel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2014635,"caller":"base/balancer.go:182","msg":"[roundrobin]roundrobinPicker: Build called with info: {map[]}","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.202357,"caller":"grpc@v1.62.1/clientconn.go:532","msg":"[core][Channel #7] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2014635,"caller":"grpc@v1.62.1/clientconn.go:1223","msg":"[core][Channel #7 SubChannel #8] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2067246,"caller":"grpc@v1.62.1/clientconn.go:1338","msg":"[core][Channel #7 SubChannel #8] Subchannel picks a new address \"localhost:14250\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2060769,"caller":"grpc@v1.62.1/clientconn.go:335","msg":"[core][Channel #7] Channel exiting idle mode","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2100976,"caller":"all-in-one/main.go:265","msg":"Starting agent"}
{"level":"info","ts":1713157066.2114675,"caller":"app/agent.go:69","msg":"Starting jaeger-agent HTTP server","http-port":5778}
{"level":"info","ts":1713157066.2114675,"caller":"grpc@v1.62.1/server.go:679","msg":"[core][Server #10] Server created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2155974,"caller":"app/static_handler.go:109","msg":"Using UI configuration","path":""}
{"level":"info","ts":1713157066.2100976,"caller":"grpc/builder.go:115","msg":"Checking connection to collector"}
{"level":"info","ts":1713157066.2100976,"caller":"sync/once.go:74","msg":"[core]CPU time info is unavailable on non-linux environments.","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2221143,"caller":"app/server.go:236","msg":"Query server started","http_addr":"[::]:16686","grpc_addr":"[::]:16685"}
{"level":"info","ts":1713157066.234199,"caller":"grpc/builder.go:131","msg":"Agent collector connection state change","dialTarget":"localhost:14250","status":"CONNECTING"}
{"level":"info","ts":1713157066.240293,"caller":"healthcheck/handler.go:129","msg":"Health Check state change","status":"ready"}
{"level":"info","ts":1713157066.240293,"caller":"app/server.go:319","msg":"Starting GRPC server","port":16685,"addr":":16685"}
{"level":"info","ts":1713157066.2465863,"caller":"grpc@v1.62.1/server.go:879","msg":"[core][Server #10 ListenSocket #12] ListenSocket created","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.240293,"caller":"app/server.go:301","msg":"Starting HTTP server","port":16686,"addr":":16686"}
{"level":"info","ts":1713157066.240293,"caller":"grpc@v1.62.1/clientconn.go:1223","msg":"[core][Channel #7 SubChannel #8] Subchannel Connectivity change to READY","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.267438,"caller":"base/balancer.go:182","msg":"[roundrobin]roundrobinPicker: Build called with info: {map[SubConn(id:8):{{Addr: \"localhost:14250\", ServerName: \"\", }}]}","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2687113,"caller":"grpc@v1.62.1/clientconn.go:532","msg":"[core][Channel #7] Channel Connectivity change to READY","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157066.2695167,"caller":"grpc/builder.go:131","msg":"Agent collector connection state change","dialTarget":"localhost:14250","status":"READY"}
{"level":"info","ts":1713157067.174541,"caller":"grpc@v1.62.1/clientconn.go:1225","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to IDLE, last error: connection error: desc = \"transport: Error while dialing: dial tcp [::1]:4317: connectex: No connection could be made because the target machine actively refused it.\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157067.1752045,"caller":"grpc@v1.62.1/clientconn.go:1223","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157067.177659,"caller":"grpc@v1.62.1/clientconn.go:1338","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"localhost:4317\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157067.180894,"caller":"grpc@v1.62.1/clientconn.go:1223","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to READY","system":"grpc","grpc_log":true}
{"level":"info","ts":1713157067.1815722,"caller":"grpc@v1.62.1/clientconn.go:532","msg":"[core][Channel #1] Channel Connectivity change to READY","system":"grpc","grpc_log":true}
{"level":"debug","ts":1713157260.8726559,"caller":"app/span_processor.go:165","msg":"Span written to the storage by the collector","trace-id":"a5409530ea95d096f1ba6732f81db8ce","span-id":"859b85e92ed972b5"}

Couldn't find how to run jaeger-all-in-one.exe with Badger database.


Solution

  • File configuration is not an actively supported mode of configuring Jaeger v1. Theoretically you could make it work by translating the documented CLI flags into their corresponding positions in the YAML file (since the library we use for CLI flags does support reading from a config file), but what you have is nowhere close to that translation.

    You can print all supported flags with this command:

    SPAN_STORAGE_TYPE=badger all-in-one print-config