apache-apisix

Apisix on Docker can not access etcd instance


I'm encountering an issue with the APISIX Docker installation. I created a custom Docker network and launched both the etcd and apisix containers within it. The etcd container is assigned the IP address 10.10.10.4, and the apisix container is on 10.10.10.5.

docker network create --subnet=10.10.10.0/24 gomath-net

ETCD:

docker run -d `
  --name etcd `
  --network gomath-net `
  --ip 10.10.10.4 `
  -p 2379:2379 `
  -p 2380:2380 `
  -e ALLOW_NONE_AUTHENTICATION=yes `
  -e ETCD_NAME=etcd1 `
  -e ETCD_ADVERTISE_CLIENT_URLS=http://10.10.10.4:2379 `
  -e ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379 `
  -e ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380 `
  -e ETCD_INITIAL_ADVERTISE_PEER_URLS=http://10.10.10.4:2380 `
  -e ETCD_INITIAL_CLUSTER="etcd1=http://10.10.10.4:2380" `
  -e ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster `
  -e ETCD_INITIAL_CLUSTER_STATE=new `
  -v etcd-data:/bitnami/etcd `
  bitnami/etcd:3.5

APISIX:

docker run -d `
  --name apisix `
  --network gomath-net `
  --ip 10.10.10.5 `
  -p 9080:9080 `
  -p 9091:9091 `
  -p 9443:9443 `
  -v ${PWD}\config.yaml:/usr/local/apisix/conf/config.yaml `
  apache/apisix:3.8.0-debian

APISIX Config (file config.yaml):

apisix:
    node_listen: 9080
    enable_admin: true
    deployment:
        admin:
            allow_admin:
                - 0.0.0.0/0
            admin_key:
                - name: admin
                  key: edd1c9f034335f136f87ad84b625c8f1
                  role: admin
        etcd:
            host:
                - "http://10.10.10.4:2379"
            prefix: /apisix
            timeout: 30  # Assuming seconds
    plugins:
        - proxy-rewrite
        - limit-req
        - prometheus

Despite updating the config.yaml file to point to http://10.10.10.4:2379 (I also tried using http://etcd:2379), the apisix container fails to start. The error indicates that it's trying to access etcd at http://127.0.0.1:2379.

It seems APISIX is not picking up the correct etcd endpoint from the configuration. What might be causing this issue?

Logs:

/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init
/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init_etcd
request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused
all etcd nodes are unavailable
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=1
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=2
/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init
/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init_etcd
request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused
all etcd nodes are unavailable
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=1
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=2

I have adjusted the config.yaml:

apisix:
  node_listen: 9080
  enable_admin: true           
plugins:                           
  - proxy-rewrite                  
  - limit-req                      
  - prometheus                     
deployment:                     
  admin:                       
    admin_key:
      - name: admin                             
        key: edd1c9f034335f136f87ad84b625c8f1  
        role: admin
    allow_admin:                  
      - 0.0.0.0/0            
  etcd:
    host:                         
      - "http://10.10.10.4:2379"
    prefix: /apisix               
    timeout: 30

However, apisix still not able to reach etcd Logs:

/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init
/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init_etcd
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=1
Warning! Request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused, retry time=2
request etcd endpoint 'http://127.0.0.1:2379/version' error, connection refused
all etcd nodes are unavailable

Solution

  • The log message Request etcd endpoint 'http://127.0.0.1:2379/version' error indicates that the configuration is wrong or that APISIX wasn't able to load it properly.

    There are basically two issues:

    1. Volume mount
    2. Wrong APISIX configuration

    1. Volume Mount

    Apparently, on Windows, there's an issue with mapping paths when binding mounts with Docker. See https://stackoverflow.com/a/64981627/19544859 for reference.

    To fix this, you can try with one of the following:

    2. APISIX Configuration

    In your config.yaml, fields deployment and plugins should not be nested under apisix, but on the same level.

    Here's the corrected version of config.yaml:

    apisix:
        node_listen: 9080
        enable_admin: true
        
    deployment:
        admin:
            allow_admin:
                - 0.0.0.0/0
            admin_key:
                - name: admin
                  key: edd1c9f034335f136f87ad84b625c8f1
                  role: admin
        etcd:
            host:
                - "http://10.10.10.4:2379"
            prefix: /apisix
            timeout: 30  # Assuming seconds
    plugins:
        - proxy-rewrite
        - limit-req
        - prometheus
    

    Useful Stuff

    You can have a look at config.yaml.example from the official GitHub repository, for a full example (with default values).