InfluxDB replication to cloud in Docker

Issue: local to cloud replication works only manually. Trying to automate.

--

Environments:

• Local in Docker (via docker-compose)
• Cloud on influxdata.com

Following this tutorial: https://docs.influxdata.com/influxdb/cloud/write-data/replication/replicate-data/#configure-a-replication-stream

Manually the replication works.

1. Build + run Docker InfluxDB
2. Open bash
3. Define influx remote and influx replication

Data reaches cloud - OK!

Automatically it fails (trying to use a bash script):

docker-compose.yml:

  influxdb2:
    image: influxdb:2
    volumes:
      - ${INFLUX_DATA_PATH}/data:/var/lib/influxdb2
      - ${INFLUX_DATA_PATH}/config:/etc/influxdb2
      - ./docker-influx/init-cloud-replication.sh:/docker-entrypoint-initdb.d/init-cloud-replication.sh
    ports:
      - "8086:8086"
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=${INFLUX_USERNAME}
      - DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUX_PASSWORD}
      - DOCKER_INFLUXDB_INIT_ORG=${INFLUX_ORG}
      - DOCKER_INFLUXDB_INIT_BUCKET=${INFLUX_BUCKET}
      - DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUX_TOKEN}
      - DOCKER_INFLUX_BUCKET_ID=${INFLUX_BUCKET_ID}

      - CLOUD_INFLUX_URL=${CLOUD_INFLUX_URL}
      - CLOUD_INFLUX_TOKEN_TELEGRAF=${CLOUD_INFLUX_TOKEN_TELEGRAF}
      - CLOUD_INFLUX_ORG_ID=${CLOUD_INFLUX_ORG_ID}
      - CLOUD_INFLUX_BUCKET=${CLOUD_INFLUX_BUCKET}
    entrypoint: [ "/bin/bash", "/docker-entrypoint-initdb.d/init-cloud-replication.sh" ]
    networks:
      - my-network

init-cloud-replication.sh:

#!/bin/bash

echo "Start InfluxDB in the background"
influxd &
INFLUX_PID=$!

# Wait for InfluxDB to be ready
while ! influx ping --host http://influxdb:8086; do
    echo "Waiting for InfluxDB to start..."
    sleep 1
done

echo "InfluxDB started and reachable."

echo "Create the remote connection"
influx remote create \
  --name my-cloud \
  --host http://influxdb:8086 \
  --token ${DOCKER_INFLUXDB_INIT_ADMIN_TOKEN} \
  --remote-url ${CLOUD_INFLUX_URL} \
  --remote-api-token ${CLOUD_INFLUX_TOKEN_TELEGRAF} \
  --remote-org-id ${CLOUD_INFLUX_ORG_ID}

REMOTE_ID=$(influx remote list --json | grep -o '"id":"[^"]*' | grep -o '[^"]*$')
if [ -z "$REMOTE_ID" ]; then
  echo "Failed to retrieve remote ID."
  exit 1
fi

echo "Create the replication service"
influx replication create \
  --name local-to-cloud \
  --org ${DOCKER_INFLUXDB_INIT_ORG} \
  --local-bucket-id ${DOCKER_INFLUX_BUCKET_ID} \
  --remote-id $REMOTE_ID \
  --remote-bucket ${CLOUD_INFLUX_BUCKET}

# Wait for influxd to terminate
wait $INFLUX_PID

The main error is:

2024-08-09 15:14:00 ts=2024-08-09T12:14:00.383700Z lvl=info msg=Listening log_id=0qulo3SW000 service=tcp-listener transport=http addr=:8086 port=8086
2024-08-09 15:14:02 OK
2024-08-09 15:14:02 InfluxDB started and reachable.
2024-08-09 15:14:02 Create the remote connection
2024-08-09 15:14:02 ts=2024-08-09T12:14:02.985721Z lvl=info msg=Unauthorized log_id=0qulo3SW000 error="authorization not found"
2024-08-09 15:14:02 ts=2024-08-09T12:14:02.994642Z lvl=info msg=Unauthorized log_id=0qulo3SW000 error="authorization not found"
2024-08-09 15:14:02 Failed to retrieve remote ID.
2024-08-09 15:14:02 Error: failed to find local org id: failed to lookup org with name "kratt_org": 401 Unauthorized: unauthorized access
2024-08-09 15:14:02 Error: failed to lookup org with name "kratt_org": 401 Unauthorized: unauthorized access 

My guess is that there's a race condition and the influx service isn't actually running properly yet, because if I re-run everything manually, it works. The official docs (and manual runs) don't even require me to specify the local host and token. For the automatic runs, they don't fix the issue:

  --host http://influxdb:8086 \
  --token ${DOCKER_INFLUXDB_INIT_ADMIN_TOKEN} \

Issue remains if I use localhost instead of influxdb as host. Google & official docs don't have a lot of answers for "influx" + "docker" + "replication"

Fix:

1. Take the Dockerfile and entrypoint.sh from the official repo. https://github.com/influxdata/influxdata-docker/tree/master/influxdb/2.7/alpine
2. Add your code (simplest: directly into the entrypoint.sh)
3. Append & to exec influxd "${@}" (result: exec influxd "${@}" &). Otherwise, no commands would be executed after this line.
4. Call your function

++ add elegance