[SOLVED] Ingesting data using ILP and Ruby

Ingesting data using ILP and Ruby

I want to send data into QuestDB using Ruby and the ILP protocol. Browsing old docs, I saw you can connect via socket and compose the messages. I tried the code below and it works, but it seems fragile as escaping columns or using the wrong time resolution sounds too easy. Do you know of any gems for this? I could always use the Postgresql protocol, but I prefer ILP for performance and for flexible schema.

require 'socket'
HOST = 'localhost'
PORT = 9009
# Returns the current time in nanoseconds
def time_in_nsec
    now = Time.now
    return now.to_i * (10 ** 9) + now.nsec
end
begin
    s = TCPSocket.new HOST, PORT
    # Single record insert
    s.puts "trades,name=client_timestamp value=12.4 #{time_in_nsec}\n"
    # Omitting the timestamp allows the server to assign one
    s.puts "trades,name=client_timestamp value=12.4\n"
    # Streams of readings must be newline-delimited
    s.puts "trades,name=client_timestamp value=12.4\n" +
            "trades,name=client_timestamp value=11.4\n"
rescue SocketError => ex
    puts ex.inspect
ensure
    s.close() if s
end

Solution

In the past QuestDB would accept ILP messages only via socket, but it has supported HTTP for a while, which means now it is compatible with Influx client libraries. You can directly use the official Influx gem and connect to QuestDB like this (note the default port is 9000, as now connection is via HTTP).

Token, bucket, and org are ignored by QuestDB, so you can pass whatever.

client = InfluxDB2::Client.new('http://localhost:9000', 'ignore-token', bucket: 'ignore-bucket', org: 'ignore-org', precision: InfluxDB2::WritePrecision::NANOSECOND, use_ssl: false)
    client.create_write_api.write(data: InfluxDB2::Point.new(name: 'table')
    .add_tag('one_symbol', 'demo')
    .add_field('amount', 2))