I want to write the data collected by the agent into different topics in the kafka according to the input plugins. Can we forward the collected data to kafka topics separately according to telegraf inputs?
This is my telegraf.conf file
[global_tags]
[agent]
interval = "20s"
round_interval = true
metric_batch_size = 10000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = "0s"
hostname = ""
omit_hostname = false
[[outputs.kafka]]
brokers = ["127.0.0.1:9092","127.0.0.1:9093"]
#topic = "telegraf"
client_id = "abc-def-ghj-klm"
max_retry = 3
## Optional TLS Config
sasl_username = "test"
sasl_password = "test123"
sasl_mechanism = "PLAIN"
insecure_skip_verify = false
data_format = "json"
[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false
[[inputs.system]]
[[inputs.internal]]
there is a simple way, just add "kafka.topic_suffix" conf to your output.kafka settings. After that you could see your new topics that was created automatically by telegraf similiar theese; telegraf_cpu, telegraf_system, telegraf_blabla.
[[outputs.kafka]]
brokers = ["127.0.0.1:9092","127.0.0.1:9093"]
topic = "telegraf"
client_id = "abc-def-ghj-klm"
max_retry = 3
## Optional TLS Config
sasl_username = "test"
sasl_password = "test123"
sasl_mechanism = "PLAIN"
insecure_skip_verify = false
data_format = "json"
[outputs.kafka.topic_suffix]
method = "measurement"
separator = "_"