I tried to install cilium with coredns in kubeadm
kube: 1.12.3 cilium: 1.3.0
I get this error:
Readiness probe failed: KVStore: Failure Err: Not able to connect to any etcd endpoints - etcd: 0/1 connected: http://127.0.0.1:31079 - context deadline exceeded
I don't know why and if i need to install etcd on the master server.
kubectl get pods -n kube-system
cilium-9z4zd 0/1 Running 3 10m
cilium-s4x2g 0/1 Running 3 10m
coredns-576cbf47c7-44hp9 1/1 Running 2 9m29s
coredns-576cbf47c7-6jst5 1/1 Running 2 9m29s
etcd-ops-kube-master-dev 1/1 Running 0 9m29s
kube-apiserver-ops-kube-master-dev 1/1 Running 0 9m29s
kube-controller-manager-ops-kube-master-dev 1/1 Running 0 9m26s
kube-proxy-79649 1/1 Running 0 38m
kube-proxy-b56fk 1/1 Running 0 38m
kube-scheduler-ops-kube-master-dev 1/1 Running 0 9m27s
I had a similar issue playing with Kubernetes the hard way, this was because of a wrong certificate
I did the following:
kubectl -n kube-system logs <etcd>
and found something like:
embed: rejected connection from "172.17.0.3:36950" (error "remote error: tls: bad certificate", ServerName "")
I got the etcd config, you should have something like
$ kubectl -n kube-system get cm cilium-config -o yaml
apiVersion: v1
data:
clean-cilium-bpf-state: "false"
clean-cilium-state: "false"
cluster-name: default
ct-global-max-entries-other: "262144"
ct-global-max-entries-tcp: "524288"
debug: "false"
disable-ipv4: "false"
etcd-config: |-
---
endpoints:
- https://<ETCD_URL>:2379
#
# In case you want to use TLS in etcd, uncomment the 'ca-file' line
# and create a kubernetes secret by following the tutorial in
# https://cilium.link/etcd-config
ca-file: '/var/lib/etcd-secrets/etcd-client-ca.crt'
#
# In case you want client to server authentication, uncomment the following
# lines and create a kubernetes secret by following the tutorial in
# https://cilium.link/etcd-config
key-file: '/var/lib/etcd-secrets/etcd-client.key'
cert-file: '/var/lib/etcd-secrets/etcd-client.crt'
legacy-host-allows-world: "false"
monitor-aggregation-level: none
sidecar-istio-proxy-image: cilium/istio_proxy
tunnel: vxlan
kind: ConfigMap
Then I compared the keys of kubectl -n kube-system get secret cilium-etcd-client-tls -o yaml that provides 3 base64 values.
I can then test the keys using curl https://<ETCD_URL>:2379/v2/keys --cacert=etcd-client-ca.crt --cert=etcd-client.crt --key=etcd-client.key
You should then have something like {"action":"get","node":{"dir":true}}
Then, you can inspect the deployment, on my side, I have
kind: Deployment
metadata:
labels:
io.cilium/app: operator
name: cilium-operator
name: cilium-operator
namespace: kube-system
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
io.cilium/app: operator
name: cilium-operator
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
io.cilium/app: operator
name: cilium-operator
spec:
containers:
- args:
- --kvstore=etcd
- --kvstore-opt=etcd.config=/var/lib/etcd-config/etcd.config
command:
- cilium-operator
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: K8S_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CILIUM_DEBUG
valueFrom:
configMapKeyRef:
key: debug
name: cilium-config
optional: true
- name: CILIUM_CLUSTER_NAME
valueFrom:
configMapKeyRef:
key: cluster-name
name: cilium-config
optional: true
- name: CILIUM_CLUSTER_ID
valueFrom:
configMapKeyRef:
key: cluster-id
name: cilium-config
optional: true
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: AWS_ACCESS_KEY_ID
name: cilium-aws
optional: true
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: AWS_SECRET_ACCESS_KEY
name: cilium-aws
optional: true
- name: AWS_DEFAULT_REGION
valueFrom:
secretKeyRef:
key: AWS_DEFAULT_REGION
name: cilium-aws
optional: true
image: docker.io/cilium/operator:latest
imagePullPolicy: Always
name: cilium-operator
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/etcd-config
name: etcd-config-path
readOnly: true
- mountPath: /var/lib/etcd-secrets
name: etcd-secrets
readOnly: true
dnsPolicy: ClusterFirst
priorityClassName: system-node-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: cilium-operator
serviceAccountName: cilium-operator
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
items:
- key: etcd-config
path: etcd.config
name: cilium-config
name: etcd-config-path
- name: etcd-secrets
secret:
defaultMode: 420
optional: true
secretName: cilium-etcd-secrets```