My Rancher setup is on-premise (vSphere). New machines get their name, DNS and IP address from vApp options parameters that are provided by a network policy profile.
I have separate node templates for worker and controlplane nodes. When I removed a worker node Rancher put one of the controlplane nodes in the worker node pool -- I now see it as both in the controlplane and worker node pools!
When I add another worker node Rancher doesn't remove the controlplane node from the worker node pool though... The controlplane doesn't have a worker role so I don't really know what to do in order to get it out of the worker node pool.
I tried adding and removing nodes via Terraform, but it made Rancher assign existing IP addresses to the new nodes, thus creating IP conflicts... In the end I have to manually remove each node from Rancher one by one until I could add more again.
UPDATE: The root cause of my problems was found! vSphere stripped vApp options from some of the machines. The vApp options parameters are used to determine available IP addresses in the network (among other things), but once they're gone new machines with existing IP addresses may be created.
Temporarily limiting the IP address range of the network and re-creating all the machines1 with disabled vApp options restored expected behavior to the cluster.
If anyone knows how vApp options can be disabled without direct interference I'd love you to share that information :)
UPDATE 2: The vApp options were removed when an ESX disconnected from its cluster. This happens because vCenter holds an entry for each ESX in its database, and once an ESX disconnects it removes that entry along with the vApp options of its VMs.
The sudden disconnections were caused by a bug in an earlier version of vCenter (6.x) and after the upgrade and the restoration of the vApp options the machines no longer suffer from missing information. (And they lived happily after, the end.)
UPDATE 3: Life became much easier when we switched from vApp options to a DHCP server. All our weird issues with the nodes stopped.
1 The vApp options can be manually restored, no need to re-create the machines (but it can be more convenient in some cases because new machines run a script that automatically populates their vApp options)