When I try to run the cockroachdb on minikube v1.29.0 on Fedora 37 (selinux disabled) with default parameters, 2 of 3 nodes fails with the following error:
Creating data directory: mkdir /cockroach/cockroach-data/auxiliary: permission denied
I don't know if it's more related to cockroachdb, minikube or my linux system. But if you have some information or hints, I'm all ears.
Thank you !!
Steps to reproduce
minikube start --cpus=2 --memory=2g --nodes=3
helm repo add cockroachdb https://charts.cockroachdb.com/
helm repo update
helm upgrade --install cockroachdb cockroachdb/cockroachdb
Results
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/cockroachdb-0 0/1 CrashLoopBackOff 1 (8s ago) 26s
pod/cockroachdb-1 0/1 CrashLoopBackOff 1 (2s ago) 26s
pod/cockroachdb-2 0/1 Running 0 26s
pod/cockroachdb-init-gkb2z 1/1 Running 0 26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cockroachdb ClusterIP None <none> 26257/TCP,8080/TCP 26s
service/cockroachdb-public ClusterIP 10.100.59.219 <none> 26257/TCP,8080/TCP 26s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6m34s
NAME READY AGE
statefulset.apps/cockroachdb 0/3 26s
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
cronjob.batch/cockroachdb-rotate-self-signer 0 0 1 */11 * False 0 <none> 26s
cronjob.batch/cockroachdb-rotate-self-signer-client 0 0 */26 * * False 0 <none> 26s
NAME COMPLETIONS DURATION AGE
job.batch/cockroachdb-init 0/1 26s 26s
$ kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
cockroachdb-0 0/1 CrashLoopBackOff 2 (15s ago) 52s 10.244.2.2 minikube-m03 <none> <none>
cockroachdb-1 0/1 CrashLoopBackOff 2 (11s ago) 52s 10.244.1.4 minikube-m02 <none> <none>
cockroachdb-2 0/1 Running 0 52s 10.244.0.3 minikube <none> <none>
cockroachdb-init-gkb2z 1/1 Running 0 52s 10.244.1.3 minikube-m02 <none> <none>
$ kubectl logs cockroachdb-0
Defaulted container "db" out of: db, copy-certs (init)
++ hostname
+ exec /cockroach/cockroach start --join=cockroachdb-0.cockroachdb.datastore.svc.cluster.local:26257,cockroachdb-1.cockroachdb.datastore.svc.cluster.local:26257,cockroachdb-2.cockroachdb.datastore.svc.cluster.local:26257 --advertise-host=cockroachdb-0.cockroachdb.datastore.svc.cluster.local --certs-dir=/cockroach/cockroach-certs/ --http-port=8080 --port=26257 --cache=25% --max-sql-memory=25% --logtostderr=INFO
Flag --logtostderr has been deprecated, use --log instead to specify 'sinks: {stderr: {filter: ...}}'.
E230321 11:03:10.501705 1 1@cli/clierror/check.go:35 [-] 1 ERROR: connection lost.
E230321 11:03:10.501705 1 1@cli/clierror/check.go:35 [-] 1 +creating data directory: mkdir /cockroach/cockroach-data/auxiliary: permission denied
ERROR: connection lost.
creating data directory: mkdir /cockroach/cockroach-data/auxiliary: permission denied
Failed running "start"
After asking ChatGPT about this, he pointed that the SecurityContext feature could be an issue: securityContext should be disabled in values.yml in the Helm chart as it prevents the writing of these files.
I still have a lot to learn about Kube though.