dockerredisredis-cli

Docker Redis - Seeding Data - Cannot execute redis-cli via Dockerfile or via shell script


Have a single Redis instance set up via docker-compose (using my Redis Dockerfile), all working.

Now I'm trying to seed the Redis instance with data after the container has started.

(1) Trying to run redis-cli directly in Dockerfile (that is run by docker-compose)

FROM redis:6.2.5
COPY ./redis.conf /test_dir/
COPY ./seed.txt /test_dir/

ENTRYPOINT ["redis-server", "/test_dir/redis.conf"]
RUN cat /test_dir/seed.txt | redis-cli -a <password> --pipe

this throws Could not connect to Redis at 127.0.0.1:6379: Connection refused

(2) Slightly different Dockerfile, running same command in a shell script that is executed after the container starts:

FROM redis:6.2.5
COPY ./redis.conf /test_dir/
COPY ./seed.txt /test_dir/
COPY ./init.sh /test_dir/

ENTRYPOINT ["redis-server", "/test_dir/redis.conf"]
RUN . /test_dir/init.sh
# init.sh
sleep 10  # ensure Redis server is up and running
cat /test_dir/seed.txt | redis-cli -a <password> --pipe

the shell script is executed, it waits 10 secs and I confirmed the server was up and running BEFORE the data import was triggered, but Redis throws the same error.

Notes:

(1) I've also disabled `protected-mode` and removed `requirepass`, but redis-cli can not connect
(2) when I jump into the container and manually execute the command, it DOES WORK !

So how can I run redis-cli from the Dockerfile or via shell script or what is the recommended way to seed a Docker Redis instance ?

Thanks.


Solution

  • RUN cat /test_dir/seed.txt | redis-cli -a <password> --pipe will execute when you do docker build, at that time ENTRYPOINT ["redis-server", "/test_dir/redis.conf"] still not run as it's only be called when the container start. So, you will surely have error.

    As a result, you could use next workaround to do it:

    /test_dir/seed.txt:

    /test_dir/init.sh:

    while :
    do
        redis-cli -h redis-svr -p 6379 quit
        if [ $? -eq 0 ]; then
            cat /test_dir/seed.txt | redis-cli -h redis-svr -p 6379 --pipe
            break
        else
            echo "server not ready, wait then retry..."
            sleep 3
        fi
    done
    

    docker-compose.yaml:

    version: '3'
    
    services:
      redis-svr:
        image: redis
      redis-cli:
        image: redis
        volumes:
          - /test_dir:/test_dir
        entrypoint: sh -c /test_dir/init.sh
    

    Execution:

    $ docker-compose up
    WARNING: Found orphan containers (20210910_redis_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
    Starting 20210910_redis-svr_1 ... done
    Starting 20210910_redis-cli_1 ... done
    Attaching to 20210910_redis-cli_1, 20210910_redis-svr_1
    redis-cli_1  | Could not connect to Redis at redis-svr:6379: Connection refused
    redis-cli_1  | server not ready, wait then retry...
    redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=1, just started
    redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.873 * monotonic clock: POSIX clock_gettime
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.879 * Running mode=standalone, port=6379.
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.879 # Server initialized
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * Loading RDB produced by version 6.2.5
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * RDB age 266 seconds
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * RDB memory usage when created 0.77 Mb
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * DB loaded from disk: 0.010 seconds
    redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * Ready to accept connections
    redis-cli_1  | OK
    redis-cli_1  | All data transferred. Waiting for the last reply...
    redis-cli_1  | Last reply received from server.
    redis-cli_1  | errors: 0, replies: 0
    20210910_redis-cli_1 exited with code 0
    

    Explain:

    redis-svr will start a server there in one container, redis-cli will start another container, it will frist call init.sh, the init.sh will try to link to redis server to see if server really start or not with redis-cli -h redis-svr -p 6379 quit. If not, it will wait sometime and retry, if server already start, then it could call the client command to import the initial data to server.

    EDIT20210912 based on OP's comment to use one container:

    Folder structure:

    $ tree
    .
    ├── docker-compose.yaml
    ├── Dockerfile
    ├── init.sh
    ├── my-entrypoint.sh
    └── seed.txt
    

    docker-compose.yaml:

    version: '3'
    
    services:
      redis-svr:
          build: ./
    

    Dockerfile:

    FROM redis:6.2.5
    COPY . /tmp
    RUN chmod -R 777 /tmp
    ENTRYPOINT ["/tmp/my-entrypoint.sh"]
    CMD ["redis-server"]
    

    init.sh:

    while :
    do
        redis-cli quit
        if [ $? -eq 0 ]; then
            echo "Server ready now, start to import data ..."
            cat ./seed.txt | redis-cli --pipe
            break
        else
            echo "Server not ready, wait then retry..."
            sleep 3
        fi
    done
    

    my-entrypoint.sh:

    #!/bin/sh
    /tmp/init.sh &
    docker-entrypoint.sh $1
    

    Execution:

    $ docker-compose up
    Recreating test_dir_redis-svr_1 ... done
    Attaching to test_dir_redis-svr_1
    redis-svr_1  | Could not connect to Redis at 127.0.0.1:6379: Connection refused
    redis-svr_1  | Server not ready, wait then retry...
    redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=7, just started
    redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
    redis-svr_1  | 7:M 12 Sep 2021 08:36:05.513 * monotonic clock: POSIX clock_gettime
    redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 * Running mode=standalone, port=6379.
    redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 # Server initialized
    redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 * Ready to accept connections
    redis-svr_1  | OK
    redis-svr_1  | Server ready now, start to import data ...
    redis-svr_1  | All data transferred. Waiting for the last reply...
    redis-svr_1  | Last reply received from server.
    redis-svr_1  | errors: 0, replies: 0
    

    This define customized entrypoint to let init.sh have chance to be executed before redis server run.