kubernetesargocdk3d

Run argocd in a k3d cluster with local repo

fascinated by k8s, I'm trying to create an argocd app that is bound to a local git repository. I think local repo is a good solution for development, I read this and I decide to try myself. I use k3d to provide cluster, here are the instructions I run:

k3d cluster create cluster1 -a 1 -v /home/myuser/workspace/argocd_tutorial:/workspace/argocd_tutorial@agent:0
kubectl create namespace argocd
kubectl apply -n argocd -f argocd/
kubectl config set-context --current --namespace=argocd
argocd app create argocd-tutorial --path "./k8s" --sync-policy auto --dest-server https://kubernetes.default.svc --repo file:///tmp/local_repo

the kubectl apply -n argocd -f argocd/ applies two .yml manifests:

#relevant parts of patched argocd installation file
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: repo-server
    app.kubernetes.io/name: argocd-repo-server
    app.kubernetes.io/part-of: argocd
  name: argocd-repo-server
spec:
....
        volumeMounts:
        ....
        - mountPath: /tmp/local_repo
          name: local-repo
          readOnly: true
      ....
      volumes:
      ....
      - name: local-repo
        persistentVolumeClaim:
          claimName: local-repo-pvc
    ....

and

#persistent volume and volume claim file
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-repo-volume
  namespace: argocd
  labels:
    type: local
spec:
  storageClassName: standard
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /workspace/argocd_tutorial
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-repo-pvc
  namespace: argocd
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: standard
  resources:
    requests:
      storage: 1Gi
  volumeName: local-repo-volume

Despite my efforts, this is what I get back:

FATA[0000] rpc error: code = InvalidArgument desc = application spec for argocd-tutorial is invalid: InvalidSpecError: repository not accessible: repositories not accessible: &Repository{Repo: "file:///tmp/local_repo", Type: "", Name: "", Project: ""}: repo client error while testing repository: rpc error: code = Unknown desc = error testing repository connectivity: repository not found

Getting a /bin/sh into argocd-repo-server I can see the /tmp/local_repo directory owned by root:

$ ls -dl /tmp/local_repo
drwxr-xr-x 2 root root 4096 Feb  6 20:38 /tmp/local_repo
$ ls -al /tmp/local_repo 
total 8
drwxr-xr-x 2 root root 4096 Feb  6 20:38 .
drwxrwxrwx 4 root root 4096 Feb  6 20:38 ..

I guess k3d mount the volume as root... Is there a way to let argocd user see that directory and all it contains as a git repo?

Solution

  • Quick fix

    The issue is that you have 1 server and 1 agent but you are mounting volume from host only to agent:0

    Just remove -a 1 and mount volume using -v /home/myuser/workspace/argocd_tutorial:/workspace/argocd_tutorial@server:0 (changed agent:0 to server:0)

    Long Answer

    At first look everything looks good, but when POD is created you don't know where it will be placed as I tried to verify it I got pod on server:0 and mounted volume is not available there.

    You can see it by calling kubectl get pods -o wide and check column NODE

    NAME                                      READY   STATUS    RESTARTS   AGE   IP          NODE                    NOMINATED NODE   READINESS GATES
echo-server-deployment-66d64444b8-g26qt   1/1     Running   0          55s   10.42.1.4   k3d-cluster1-server-0   <none>           <none>

    As you can see in my case it's server:0 when I scaled deployment to 2 replicas second one appeared on agent:0 and all data is available on that POD.

    You can solve it 2 ways

    I think second is better because everything is for testing purpose, so until you really need more then one Node using just one server without agents is perfectly fine as everything is anyway on one machine and server is also running PODs.