questdb

Duplicate records when using Influx line protocol with QuestDb


I am using QuestDb and posting record with like

influxDB = InfluxDBFactory.connect("http://localhost:9000", username, password);
influxDB.setDatabase(database);
influxDB.enableBatch(BatchOptions.DEFAULTS);
influxDB.write(Point.measurement(TABLE_NAME)
                .addField("ID",ir.getid())
                .addField(....)
                .build());

We do not send timestamp, so QuestDb inserts the server time and running it with one worker thread server option.

Each ID is unique but when I query QuestDb I see sometimes 5 records with the same ID as if QuestDb creates duplicates.

What can be wrong here?

The duplicates have all the same values, except timestamp is roughly 10s apart from one to another.


Solution

  • QuestDb does not support HTTP connection, it support TCP instead.

    What probably happens is that you open HTTP connection to the port, it opens underlying TCP socket connection and sends HTTP headers, QuestDb ignores the headers as invalid messages and parses valid Influx protocol lines. Then Influx library does not receives any response and re-sends the message again after some configured interval, and again... Here you got the duplicates

    Switch to TCP connection and do not use Influx library to send the messages using something like https://questdb.io/docs/develop/insert-data/ or telegraf, or use UDP