kubernetesyamlkubectlbare-metaldgraph

How to run Dgraph on bare-metal Kubernetes cluster


I am trying to setup Dgraph in HA Cluster but it won't deploy if no volumes are present.

When directly applying the provided config on a bare-metal cluster won't work.

$ kubectl get pod --namespace dgraph
dgraph-alpha-0                      0/1     Pending     0          112s
dgraph-ratel-7459974489-ggnql       1/1     Running     0          112s
dgraph-zero-0                       0/1     Pending     0          112s


$ kubectl describe pod/dgraph-alpha-0 --namespace dgraph
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  error while running "VolumeBinding" filter plugin for pod "dgraph-alpha-0": pod has unbound immediate PersistentVolumeClaims
  Warning  FailedScheduling  <unknown>  default-scheduler  error while running "VolumeBinding" filter plugin for pod "dgraph-alpha-0": pod has unbound immediate PersistentVolumeClaims

Anyone else has this problem? I've been experiencing this issue for several days now and can not find a way around this. How can I have Dgraph use cluster's local storage?

Thanks


Solution

  • Found a working solution myself.

    I have to manually create the pv and pvc, then Dgraph can use them during deployment.

    Here is the config I used to create the needed storageclass, pv and pvc

    ---
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: local
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
    provisioner: kubernetes.io/no-provisioner
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-alpha-0
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/alpha-0"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-alpha-1
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/alpha-1"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-alpha-2
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/alpha-2"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-zero-0
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/zero-0"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-zero-1
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/zero-1"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: datadir-dgraph-dgraph-zero-2
      labels:
        type: local
    spec:
      storageClassName: local
      capacity:
        storage: 8Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/mnt/dgraph/zero-2"
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-alpha-0
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-alpha-1
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-alpha-2
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-zero-0
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-zero-1
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: datadir-dgraph-dgraph-zero-2
    spec:
      storageClassName: local
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
    

    When Dgraph is deployed it latches at the pvc

    $ kubectl get pvc -n dgraph -o wide
    NAME                            STATUS   VOLUME                          CAPACITY   ACCESS MODES   STORAGECLASS   AGE     VOLUMEMODE
    datadir-dgraph-dgraph-alpha-0   Bound    datadir-dgraph-dgraph-zero-2    8Gi        RWO            local          6h40m   Filesystem
    datadir-dgraph-dgraph-alpha-1   Bound    datadir-dgraph-dgraph-alpha-0   8Gi        RWO            local          6h40m   Filesystem
    datadir-dgraph-dgraph-alpha-2   Bound    datadir-dgraph-dgraph-zero-0    8Gi        RWO            local          6h40m   Filesystem
    datadir-dgraph-dgraph-zero-0    Bound    datadir-dgraph-dgraph-alpha-1   8Gi        RWO            local          6h40m   Filesystem
    datadir-dgraph-dgraph-zero-1    Bound    datadir-dgraph-dgraph-alpha-2   8Gi        RWO            local          6h40m   Filesystem
    datadir-dgraph-dgraph-zero-2    Bound    datadir-dgraph-dgraph-zero-1    8Gi        RWO            local          6h40m   Filesystem