I want to build a python script that accesses a kubernetes worker node, lists the network interfaces and captures the traffic of the choosen interface (with tcpdump for exemple) and then store the pcap file somewhere on the master node.
I want to know if it is possible to access a worker node from my master node without ssh? (maybe with a direct k8s-apiserver call ?) If ssh is the only way to access the worker node, how can i make a connection without entring the worker's password (for authentification).
Or maybe there is another way to do this ?
One way to connect nodes, when SSH isn't an option, would be to start some privileged container, that would access your nodes filesystem, disabling pid namespace isolation.
Say I have some node "infra1"
$> kubectl get nodes
infra1 Ready infra 728d v1.21.6
I can get in using:
$ kubectl debug node/infra1 -it --image=busybox
Creating debugging pod node-debugger-infra1-s46g6 with container debugger on node infra1.
If you don't see a command prompt, try pressing enter.
/ #
/ # chroot /host
root@infra1:/#
root@infra1:/# crictl ps | head -2
CONTAINER IMAGE CREATED STATE
NAME ATTEMPT POD ID
a36863efe2a2e 3fb5cabb64693 4 minutes ago Running
The /host
being a "volume", sharing my host filesystem with that debug container. Using chroot, you're now working from your node runtime.
$ k get pods
NAME READY STATUS RESTARTS AGE
node-debugger-infra1-g5nwg 0/1 Error 0 71s
node-debugger-infra1-s46g6 1/1 Running 0 55s
In practice, this is done creating a Pod, such as the following:
apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/psp: hostaccess
name: node-debugger-infra1-s46g6
namespace: default
spec:
containers:
- image: busybox
...
volumeMounts:
- mountPath: /host
name: host-root
...
dnsPolicy: ClusterFirst
enableServiceLinks: true
hostIPC: true
hostNetwork: true
hostPID: true
nodeName: infra1
nodeSelector:
kubernetes.io/os: linux
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- operator: Exists
volumes:
- hostPath:
path: /
type: ""
name: host-root
...
Answering your follow-up question, regarding caliXXX interfaces. Those are specific to calico SDN, although the same remarks may apply to other implementations: there's no easy way to resolve pods IPs from those.
We can however inspect pods configuration, figuring out which interface they use:
# crictl pods
...
b693b9ff6487c 3 hours ago Ready controller-6b78bff7d9-5b7vr metallb-system 2 (default)
# crictl inspectp b693b9ff6487c | jq '.info.cniResult'
{
"Interfaces": {
"cali488d4aeb0e6": {
"IPConfigs": null,
"Mac": "",
"Sandbox": ""
},
"eth0": {
"IPConfigs": [
{
"IP": "10.233.105.55",
"Gateway": ""
}
],
"Mac": "",
"Sandbox": ""
},
"lo": {
"IPConfigs": [
{
"IP": "127.0.0.1",
"Gateway": ""
},
{
"IP": "::1",
"Gateway": ""
}
],
"Mac": "00:00:00:00:00:00",
"Sandbox": "/var/run/netns/cni-55322514-e37a-2913-022a-9f7488df8ca5"
}
},
"DNS": [
{},
{}
],
"Routes": null
}
Then, resolving the interface name for a given IP, we could so something like:
# MATCH_IP="10\.233\.105\.55"
# crictl pods | awk '/Ready/{print $1}' | while read pod
do
crictl inspectp $pod | grep $MATCH_IP >/dev/null 2>&1 || continue
echo found pod $pod
crictl inspectp $pod \
| jq '.info.cniResult.Interfaces | with_entries(select(.key|match("cali"))) | to_entries[] | .key'
break
done
found pod b693b9ff6487c
"cali488d4aeb0e6"