argocd

ArgoCD: transport: Error while dialing dial tcp: lookup argocd-repo-server

I am configuring ArgoCD, and all the pods are Running state like below.

$ kubectl get pods -n argocd -o wide
NAME                              READY   STATUS    RESTARTS   AGE    IP               NODE        NOMINATED NODE   READINESS GATES
argocd-application-controller     1/1     Running   0          138m   172.16.195.218   worker-1    <none>           <none>
argocd-applicationset-controller  1/1     Running   0          138m   172.16.195.216   worker-1    <none>           <none>
argocd-dex-server                 1/1     Running   0          138m   172.16.59.213    worker-2    <none>           <none>
argocd-notifications-controlle    1/1     Running   0          138m   172.16.195.217   worker-1    <none>           <none>
argocd-redis                      1/1     Running   0          138m   172.16.59.214    worker-2    <none>           <none>
argocd-repo-server                1/1     Running   0          46m    172.16.59.216    worker-2    <none>           <none>
argocd-server                     1/1     Running   0          138m   172.16.59.215    worker-2    <none>           <none>

But when I create a new app, ArgoCD shows the following error.

Unable to create application: application spec for test is invalid: InvalidSpecError: repository not accessible: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup argocd-repo-server on 10.96.0.10:53: read udp 172.16.59.215:50498->10.96.0.10:53: i/o timeout"

This error occurs when not only using private git repository but also public github repository. And curl to git repository from worker-2 node is OK.

Maybe it seems that the connection from argocd-server to argocd-repo-server is timed out. But I cannot understand why this problem occurs.

My Environment:

Solution

  • According to your logs you've probably faced IP connectivity issue with DNS server. Apparently, due to inability to resolve domain names argocd-server cannot initiate connection to argocd-repo-server.

    A general plan to you how to troubleshoot such issues:

    1. be sure that your DNS pod is up & running in your K8s cluster.
    2. be sure your pod has IP connectivity with K8s DNS server.
    3. be sure your pod has access to UDP/53 of K8s DNS server.
    4. be sure DNS entry that your pod's asking for is resolved, so argocd-repo-server is resolved to an IP address.

    See that pretty explained guide for more details - kubernetes.io.