While using Kubernetes API, I came across this issue where I am not able to pod exec into another pod using the API.
I made sure I am running a pod with a serviceAccount that has correct RBAC permissions (and both pods are in the same namespace). I tried with curl (7.74.0) but I stumbled upon this SO answer that curl may not be the right utility to do this. I have already tried other possible options (this and this) mentioned in the comments in the same post, but no luck.
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
NAMESPACE=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)
curl -k -H "Upgrade: SPDY/3.1" -H "Authorization: Bearer $TOKEN" -H "Connection: Upgrade" -H "Sec-WebSocket-Version: 13" -H "Accept: application/json" "https://kuberne
tes.default.svc/api/v1/namespaces/$NAMESPACE/pods/nginx-deployment-5c547569b4-jgm85/exec?command=ls"
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Upgrade request required",
"reason": "BadRequest",
"code": 400
}
Have already tried adding/removing few of those headers.
The next thing I tried was wscat (5.2.0). Tried a couple of variations, but needless to say, "no luck" again.
wscat -n -H "Authorization: Bearer $TOKEN" -c wss://kubernetes.default.svc/api/v1/namespaces/$NAMESPACE/pods/nginx-deployment-5c547569b4-jgm85/exec?command=ls
error: Unexpected server response: 403
This command outputs nothing (at all):
wscat -c wss://kubernetes.default.svc/api/v1/namespaces/$NAMESPACE/pods/nginx-deployment-5c547569b4-jgm85/exec?command=ls&stdin=true&stdout=true&tty=true&token=$TOKEN&namespace=$NAMESPACE
I looked at this SO post, but it doesn't work for me.
wscat -n -H "Authorization: Bearer $TOKEN" -c 'https://kubernetes.default.svc/api/v1/namespaces/$NAMESPACE/pods/nginx-deployment-5c547569b4-jgm85/exec?command=ls&stdin=true&stdout=true&tty=true'
...
SyntaxError: The URL's protocol must be one of "ws:", "wss:", or "ws+unix:"
...
Not sure how the fella even managed to use http(s) in the first place.
Can someone kindly advise if there's something wrong with what I doing or an alternative (can't use kubectl) to do the same? TIA
Curl Magic and trial and error. Make sure that the user has the right permissions so you don't get the 403.
curl -k \
--http1.1 \
-H "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
-H "Sec-WebSocket-Version: 13" \
-i \
-N \
-L \
-k \
-H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Authorization: Bearer $TOKEN" \
"https://127.0.0.1:53329/api/v1/namespaces/default/pods/single-wrong-arch/exec?command=ls&command=-l&stdin=true&stdout=true&stderr=true"
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: qGEgH3En71di5rrssAZTmtRTyFk=
Sec-WebSocket-Protocol:
total 56
~drwxr-xr-x 2 root root 4096 Mar 29 14:45 bin
drwxr-xr-x 5 root root 360 Apr 27 11:03 dev
drwxr-xr-x 1 root root 4096 Apr 27 11:03 etc
drwxr-xr-x 2 root root 4096 Mar 29 14:45 home
drwxr-xr-x 7 root root 4096 Mar 29 14:45 lib
drwxr-xr-x 5 root root 4096 Mar 29 14:45 media
drwxr-xr-x 2 root root 4096 Mar 29 14:45 mnt
drwxr-xr-x 2 root root 4096 Mar 29 14:45 opt
dr-xr-xr-x 248 root root 0 Apr 27 11:03 proc
drwx------ 2 root root 4096 Mar 29 14:45 root
drwxr-xr-x 1 root root 4096 Apr 27 11:03 run
drwxr-xr-x 2 root root 4096 Mar 29 14:45 sbin
drwxr-xr-x 2 root root 4096 Mar 29 14:45 srv
dr-xr-xr-x 13 root root 0 Apr 27 11:03 sys
drwxrwxrwt 2 root root 4096 Mar 29 14:45 tmp
drwxr-xr-x 7 root root 4096 Mar 29 14:45 usr
drwxr-xr-x 12 root root 4096 Mar 29 14:45 var
That's against a minikube cluster, which should explain the port choice and here's the rest of the environemt:
$ curl --version
curl 7.87.0 (x86_64-apple-darwin22.0) libcurl/7.87.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0
Release-Date: 2022-12-21
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets
$ minikube version
minikube version: v1.30.1
commit: 08896fd1dc362c097c925146c4a0d0dac715ace0
$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.0", GitCommit:"1b4df30b3cdfeaba6024e81e559a6cd09a089d65", GitTreeState:"clean", BuildDate:"2023-04-11T17:04:23Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-15T13:33:12Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/arm64"}
$ uname -a
Darwin 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar 6 21:00:41 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103 arm64