I'm developing this dummy project and trying to make it work locally via Skaffold
.
There are 3 services in my project (running on ports 3001
, 3002
and 3003
respectively), wired via NATS server
.
The problem is: I get different kinds of errors each time I run skaffold debug
, and one/more service(s) don't work.
At times, I don't get any errors, and all services work as expected. The followings are some of the errors:
Waited for <...>s due to client-side throttling, not priority and fairness,
request: GET:https://kubernetes.docker.internal:6443/api/v1/namespaces/default/pods?labelSelector=app%!D(MISSING). <...>%!C(MISSING)app.kubernetes.io%!F(MISSING)managed-by%!D(MISSING)skaffold%!C(MISSING)skaffold.dev%!F(MISSING)run-id%!D(MISSING)<...>` (from `request.go:668`)
- `0/1 nodes are available: 1 Insufficient cpu.` (from deployments)
- `UnhandledPromiseRejectionWarning: NatsError: CONNECTION_REFUSED` (from apps)
- `UnhandledPromiseRejectionWarning: Error: getaddrinfo EAI_AGAIN nats-service` (from apps)
I'm at a loss and can't help myself anymore. I hope someone here will be able to help me out.
Thanks in advance.
PS: Below is my machine's config, in case it's my machine's fault.
Processor: AMD Ryzen 7 1700 (8C/16T)
Memory: 2 x 8GB DDR4 2667MHz
Graphics: AMD Radeon RX 590 (8GB)
OS: Windows 10 Pro 21H1
$ docker version
Client:
Version: 19.03.12
API version: 1.40
Go version: go1.13.12
Git commit: 0ed913b8-
Built: 07/28/2020 16:36:03
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:52:10 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b63
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.4", GitCommit:"3cce4a82b44f032d0cd1a1790e6d2f5a55d20aae", GitTreeState:"clean", BuildDate:"2021-08-11T18:10:22Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
I use WSL2 (Debian) and docker-desktop
is the context of Kubernetes.
The main reason of issues like this one is that you are setting only CPU limit (without setting CPU request) so Kubernetes automatically assigns a CPU request which is equal to the CPU limit:
If you specify a CPU limit for a Container but do not specify a CPU request, Kubernetes automatically assigns a CPU request that matches the limit. Similarly, if a Container specifies its own memory limit, but does not specify a memory request, Kubernetes automatically assigns a memory request that matches the limit.
So as requests are equal to the limits, your node can't meet these requirements (you have 16 CPUs available; to start all services you need 24 CPUs) - that's why you are getting 0/1 nodes are available: 1 Insufficient cpu
error message.
How to fix it?
But...
You wrote that:
Should I also try setting up the
requests
key and set the lower limit too? Or what about completely omitting it? I tried that one, and still same issue.
So if you deleted all CPU limits from all deployments and you still have error related to the insufficient CPU it clearly shows that your app is too resource-hungry. I'd suggest optimizing the application in terms of resource utilization. Another option is to increase node resources.