I’m using InfluxDB 2.0 in Docker along with a custom notification receiver (influx-adapter-api container) that writes to InfluxDB using the Python influxdb-client in synchronous mode.
So it seems likely the issue is with the InfluxDB client or the runtime inside the influx-adapter-api container.
What I’ve Tried
Question:
What could cause the InfluxDB Python client or the influx-adapter-api container to silently stop writing to InfluxDB after running for some time?
Could it be a client session leak, open connection timeout, thread hang, or socket exhaustion?
Any known issues or recommended workarounds for long-lived InfluxDB write clients in Docker?
influxdb:
image: influxdb:2.0
restart: unless-stopped
expose:
- "8086"
ports:
- "8086:8086" # For testing only
environment:
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=influxdb
- DOCKER_INFLUXDB_INIT_PASSWORD=influxdb
- DOCKER_INFLUXDB_INIT_ORG=Polito
- DOCKER_INFLUXDB_INIT_BUCKET=test
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=password
volumes:
- influxdb_data:/var/lib/influxdb2
writer container:
influx-adapter-api:
image: influx-adapter-api:latest
restart: unless-stopped
expose:
- "8008"
volumes:
- setting:/influx-adapter-api
the code:
from influxdb_client import InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS
from datetime import datetime
import time, threading, logging
class InfluxWriter:
def __init__(self, url, org, bucket, username, password):
self.client = InfluxDBClient(url=url, username=username, password=password, org=org)
self.write_api = self.client.write_api(write_options=SYNCHRONOUS)
threading.Thread(target=self._periodic_reconnect, daemon=True).start()
def _periodic_reconnect(self):
while True:
time.sleep(3600)
self._reconnect()
def _reconnect(self):
self.write_api.close()
self.client.close()
self.client = InfluxDBClient(url=self.url, username=self.username, password=self.password, org=self.org)
self.write_api = self.client.write_api(write_options=SYNCHRONOUS)
def write_notification_to_influxdb(self, measurement, fields, timestamp):
data = {"measurement": measurement, "fields": fields, "time": timestamp}
try:
self.write_api.write(bucket=self.bucket, record=data)
self.write_api.flush()
except Exception as e:
if "session not found" in str(e):
self._reconnect()
self.write_api.write(bucket=self.bucket, record=data)
self.write_api.flush()
else:
logging.error(f"Write error: {e}")
I tried using a token instead of user/pass, and it's been working longer, but I need to keep an eye on it. However, this does not explain why my other services are working; only this one is disconnecting and is not showing any errors.
First, I created a token with
influx auth create \
--org <ORG_NAME> \
--read-buckets \
--write-buckets \
--description "Token for My App"
and then
InfluxDBClient(url="http://localhost:8086", token="my-token", org="my-org")