mqttinfluxdbtelegraf

Telegraf uses timestamp when data sent to Influx, not when data received?

I have a telegraf instance, which receives it's data solely from MQTT on the same device as telegraf. It then outputs the data to InfluxDB on a separate server. The network connection to the InfluxDB server is not reliable and sometimes goes down for an extended period of time. (Minutes to hours)

Thus, I have configured telegraf to buffer this data using the metric_buffer_limit parameter, which does work.

HOWEVER, it appears that InfluxDB sees these data points as having occurred when it receives them from telegraf, not when telegraf receives the actual data from MQTT. I was wondering if I need some parameter added to my telegraf configuration to achieve this.

Is there a way to make these data points be properly associated with the point in time when the information was received?

My telegraf.conf:

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 100
  metric_buffer_limit = 50000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "1s"
  debug = false
  quiet = false
  hostname = "{snip}"
  omit_hostname = false

################################
#            OUTPUTS           #
################################

#Sensors DB
[[outputs.influxdb_v2]]
  urls = ["{snip}"]
  token = "{snip}"
  organization = "{snip}"
  bucket = "{snip}"

################################
#            INPUTS            #
#            SENSORS           #
################################

[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  qos = 0
  connection_timeout = "30s"
  topics = [ "#" ]
  client_id = "telegraf"
  persistent_session = false
  data_format = "json"
Solution

  • Do the data points from MQTT have a timestamp themselves? If so, that timestamp can be used to set the data's timestamp rather than the current time.

    If not, another option would be to use a starlark processor to set the timestamp during processing.

    Edit: @giovanni-luisotto got me to look, and it appears the mqtt input uses a TrackingAccumulator. This special accumulator in Telegraf ensures that metrics are captured, queued, and successfully sent to outputs. The timestamp is set when the data is set to outputs, I believe, unless it is already set.