[SOLVED] Can't launch pod using gVisor

Can't launch pod using gVisor

I'm trying to install the gvisor addon in minikube: https://github.com/kubernetes/minikube/blob/master/deploy/addons/gvisor/README.md

minikube start --container-runtime=containerd  \
    --docker-opt containerd=/var/run/containerd/containerd.sock
minikube addons enable gvisor

After a short wait, the gvisor pod is running

NAME         READY   STATUS    RESTARTS   AGE
pod/gvisor   1/1     Running   0          24s

So far, so good. But when I try to create the example pod, it stays stuck in ContainerCreating

Events:
  Type     Reason                  Age   From               Message
  ----     ------                  ----  ----               -------
  Normal   Scheduled               55s   default-scheduler  Successfully assigned default/nginx-untrusted to minikube
  Warning  FailedCreatePodSandBox  50s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "75807f2be807da0264c5210cba355a294bd8725ca29ea565b05685cb5fa4ddee": failed to set bridge addr: could not add IP address to "cni0": permission denied
  Warning  FailedCreatePodSandBox  38s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "47e315477fd91cd2a542e98f26fed5e2e758b8655c298048dcb3b2aa1cb47a49": failed to set bridge addr: could not add IP address to "cni0": permission denied
  Warning  FailedCreatePodSandBox  22s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "be59a5452249bec42f5400f7465de22e9c91cd35b9a492673b7215dc3097571d": failed to set bridge addr: could not add IP address to "cni0": permission denied
  Warning  FailedCreatePodSandBox  6s    kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4fd2de889a5c685dfe590e8ed4d86b66ce3a11c28cfa02eb42835b2b3b492723": failed to set bridge addr: could not add IP address to "cni0": permission denied

Searching for what might be causing the permission denied message hasn't turned up anything useful. I have tried forcing differnt CNI options (bridge, calico), but these just lead to other errors. If I disable gvisor, I can create pods without any issues, so the containerd runtime seems to be working fine.

Any tips on how to track down where the "permission denied" message is coming from would be appreciated. minikube logs just seems to repeat the same "permission denied" message.

Solution

I raised an issue and here is the response:

https://github.com/google/gvisor/issues/7877#issuecomment-1226399080

containerd has updated its configuration format (once again). Minikube breaks because the plugin is trying to use the old format. Let me do a quick fix for now...we'll need a better way to patch config.toml to configure the runtime. Right now, it replaces the entire file and may lose other configuration changes.

In short, it should be patched in the next release.