open-telemetryopen-telemetry-collector

set Timestamp field on log data model in opentelemetry collector using transform processor


I've configured my OpenTelemetry Collector (otel/opentelemetry-collector-contrib:0.82.0) to receive events via the FluentForward receiver. The incoming events have a structure like this:

{
  "message": "hello world",
  "timestamp": "1692092369949725000"
}

My goal is to modify the final OTLP payload before it's sent to the exporter, so that it contains my message timestamp (which is in epoch format in nanoseconds) rather than the ingested time.

Here's the configuration I've tried:

receivers:
  fluentforward:
    endpoint: 0.0.0.0:8006

processors:
  transform/example:
    log_statements:
      - context: log
        statements:
          - set(timestamp, attributes["timestamp"])

exporters:
  logging:
    loglevel: debug

service:
  pipelines:
    logs:
      receivers: [fluentforward]
      processors: [transform/example]
      exporters: [logging]

However, I'm encountering an error:

Error: invalid configuration: processors::transform/example: unable to parse OTTL statement "set(timestamp, attributes["timestamp"])". Error details: error while parsing arguments for call to "set": invalid argument at position 0: invalid path expression [{timestamp []}]


Solution

  • The path timestamp is not a valid path for the logs context.
    All valid paths can be found here.
    Also, your timestamp attribute appears to be a string, so you'll need to convert it to an int.

    The following statement should work:

    set(time_unix_nano, Int(attributes["timestamp"]))