I am attempting to run a k3d cluster on a linux ubuntu machine using the following config:
{
"clusterToken": "123123123132",
"imageVolume": "k3d-tl-images",
"InitNode": null,
"kubeAPI": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "43077"
},
"host": "0.0.0.0",
"Port": "6443"
},
"kubeconfig": {
"switchCurrentContext": true,
"updateDefaultKubeconfig": true
},
"name": "tl",
"network": {
"id": "123123123123123",
"ipam": {
"ipPrefix": "172.19.0.0/16",
"ipsUsed": [
"172.19.0.1",
"172.19.0.3"
],
"Managed": false
},
"Members": [
{
"IP": "172.19.0.3",
"Name": "k3d-tl-registry"
}
],
"name": "k3d-tl"
},
"nodes": [
{
"agentOpts": {
},
"Cmd": [
"server"
],
"created": "2023-07-16T09:08:47.253631752Z",
"env": [
"all_proxy=",
"ALL_PROXY=",
"http_proxy=",
"HTTP_PROXY=",
"https_proxy=",
"HTTPS_PROXY=",
"no_proxy=",
"NO_PROXY=",
"K3S_TOKEN=123123123123",
"K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml"
],
"extraArgs": [
"--disable=traefik",
"--tls-san",
"0.0.0.0"
],
"ExtraHosts": null,
"GPURequest": "",
"image": "docker.io/rancher/k3s:v1.21.7-k3s1",
"IP": {
"IP": "",
"Static": false
},
"Memory": "",
"name": "k3d-tl-server-0",
"Networks": [
"k3d-tl"
],
"portMappings": {
"6443": [
{
"HostIp": "0.0.0.0",
"HostPort": "43077"
}
],
"80/tcp": [
{
"HostIp": "",
"HostPort": "4589"
}
]
},
"restart": true,
"role": "server",
"runtimeLabels": {
"app": "k3d",
"k3d.cluster": "tl",
"k3d.cluster.imageVolume": "k3d-tl-images",
"k3d.cluster.network": "k3d-tl",
"k3d.cluster.network.external": "true",
"k3d.cluster.network.id": "123123123123123",
"k3d.cluster.network.iprange": "172.19.0.0/16",
"k3d.cluster.token": "hjeriYqaurJuGhjUxVrg",
"k3d.cluster.url": "https://k3d-tl-server-0:6443",
"k3d.role": "server",
"k3d.server.api.host": "0.0.0.0",
"k3d.server.api.hostIP": "0.0.0.0",
"k3d.server.api.port": "43077",
"k3d.server.loadbalancer": "",
"k3d.version": "v5-dev"
},
"serverOpts": {
"kubeAPI": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "43077"
},
"host": "0.0.0.0",
"Port": "6443"
}
},
"State": {
"Running": true,
"Started": "2023-07-16T09:08:47.969985394Z",
"Status": ""
},
"volumes": [
"/var/lib/tl/standalone:/var/lib/tl/standalone",
"/home/tl/tl/data:/home/tl/tl/data",
"k3d-tl-images:/k3d/images"
]
}
],
"options": {
"disableLoadbalancer": true,
"globalEnv": [
"K3S_TOKEN=123123123"
],
"globalLabels": {
"app": "k3d",
"k3d.cluster": "tl",
"k3d.cluster.imageVolume": "k3d-tl-images",
"k3d.cluster.network": "k3d-tl",
"k3d.cluster.network.external": "true",
"k3d.cluster.network.id": "123123123123",
"k3d.cluster.network.iprange": "172.19.0.0/16",
"k3d.cluster.token": "123123123123",
"k3d.cluster.url": "https://k3d-tl-server-0:6443",
"k3d.server.loadbalancer": ""
},
"nodeHooks": [
{
"action": {
"Content": "123123123123123123",
"Description": "Write LocalRegistryHosting Configmap",
"Dest": "/tmp/localRegistryHostingCM.yaml",
"Mode": 420,
"Runtime": {
}
},
"stage": "preStart"
},
{
"action": {
"Content": "123123123123123123123123",
"Description": "Write Registry Configuration",
"Dest": "/etc/rancher/k3s/registries.yaml",
"Mode": 420,
"Runtime": {
}
},
"stage": "preStart"
}
],
"registries": {
"config": {
"Auths": null,
"Configs": null,
"Mirrors": {
"docker.elastic.co": {
"Endpoints": null,
"Rewrites": null
},
"docker.io": {
"Endpoints": null,
"Rewrites": null
},
"gcr.io": {
"Endpoints": null,
"Rewrites": null
},
"k8s.gcr.io": {
"Endpoints": null,
"Rewrites": null
},
"quay.io": {
"Endpoints": null,
"Rewrites": null
},
"us-central1-docker.pkg.dev": {
"Endpoints": null,
"Rewrites": null
}
}
},
"use": [
{
"ClusterRef": "",
"expose": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "5699"
},
"Port": "5000/tcp"
},
"host": "k3d-tl-registry",
"image": "sha256:123123123123123123123",
"options": {
"proxy": {
"remoteURL": ""
}
}
}
]
},
"waitForServer": true
},
"volumes": [
"k3d-tl-images"
]
}
And the command (golang):
k3dCluster.ClusterGet(ctx, runtimes.SelectedRuntime, &clusterConfig.Cluster)
Which results in the following error:
Failed Cluster Start: error during post-start cluster preparation: error overwriting contents of /etc/hosts: docker failed to attach to exec process in node 'k3d-tl-server-0': http: invalid Host header
I have tried manually setting permissions for the etc/hosts file, ensuring its valid, updating the http_proxy to be localhost and 127.0.0.1 and so on while always getting the same issue.
I see in the logs that the command docker is attempting to run when it fails with that error is:
Executing command '[sh -c cat /tmp/-etc-hosts-123123 > /etc/hosts]' in node 'k3d-tl-server-0'
Any assistance with understanding the cause and solution would be much appreciated
http: invalid Host header
It could be this bug in docker.
To check, if it's the case, try doing docker exec to attach to a running container. Should you see that error, it's that bug.
If it is the case: either downgrade, or wait until a fix is released
(by the operating system maintainers, or maintainers of an external repository, whatever applies).
I have this error with my current docker (24.0.4-ce built with go1.20.6) when using the "docker exec" command.