I'm running grafana mimir in docker and tried clustering it with the members being discovered by a DNS query, but unfortunately, it keeps telling me
caller=resolver.go:87 level=error msg="failed to lookup IP addresses" host=tasks.monitoring_mimir err="lookup tasks.monitoring_mimir on 127.0.0.11:53: no such host"
caller=resolver.go:134 level=warn msg="IP address lookup yielded no results. No host found or no addresses found" host=tasks.monitoring_mimir
As you can see on the screenshot, the query returns the correct IPs (they are pingable and checking the network shows that those are, in fact, the right ones) and also uses the docker dns server running on 127.0.0.11:53
Also, when running the dig
query as the entrypoint, it still resolves correctly, therefore I doubt that it only works because it takes me some time to execute the query
The configs used can be found in this gist, I deploy the stack with docker deploy --compose-file docker-compose.yml monitoring
Unfortunately, this seems to still be an issue to this day.
However, there is a workaround: manually overriding the IP used for discovery