We migrated our 3 Solr servers to 3 new VMs. We are still running the same setup as we were on the old VMs (Solr 7.4). I've also ran the Zookeeper upconfig command to replace our old config files so that they now use the new IPs. However, when I view the Solr Cloud UI, 2 of our old IPs are being shown on the Cloud > Graph. I verified that the Zookeeper upconfig worked because I can see that the new configset files for my collection are there in the cloud > tree > configs. They are also showing the new IPs in the files. So I'm not sure why the Cloud > Graph is showing 2 of our old IPs. Also, when I checked the logs, I see the following error: null:org.apache.solr.common.SolrException: Error trying to proxy request for url: http://139.XX.XX.34:8983/solr/MyCollection/select
The IP that the error is mentioning is the IP for one of the old VMs. There is also another error message like that for the other old IP. Any ideas where it could be getting the IP value from? I thought that the Zookeeper upconfig would've fixed this and I've searched all of my solr and zookeeper files to see if there was a config file I missed, but didn't find any that mentioned the old IPs anywhere.
You need to remove the old servers replica from Solr cluster if they still appear in Solr UI.
You can remove them from Solr UI under collections menu. There's a red cross button for each replica.