Today I am facing a problem that the redis in the kubernetes cluster shows error like this:
2021-11-16T05:32:18 [INFO] - increment delete failed...,MobcError(RedisCMDError(An error was signalled by the server: You can't write against a read only replica.))
I check my redis config and found that the config url like this:
redisConnectionStr="redis://default:password@cruise-redis-headless.reddwarf-cache.svc.cluster.local:6379/3"
I check the servcie and found it mapped to the redis cluster master and replica, I was wonder the headness service user for what situation? why not only keep the master servcie and replica servcie? If read operate, use any of the servcie would be fine. If write, only use the master service. when should i use the headness service?
why not only keep the master service and replica servcie?
You can do it, if you could implement it by your own.
As redis helm maybe dont use specific labels to run Read/Write replicas so it's hard to divert and create the different services for read & write.
If you are deploying the Redis using the helm it will create the Two services into the K8s cluster
Headless service
ClusterIP service
In the your application code side, you should be using the normal clusterIP service.
So idea is headless return the pod replicas IPs which will be used further by the Redis internally to manage the cluster if you are using the Redis sentinel or Redis cluster.
If you are using the ClusterIP service it will auto route the traffic to Read/Write replicas based on the condition.
If you are not using sentinel or cluster simply use ClusterIP and you are good to go.
Sentinel details
If you are using the Sentinel feature, you can ping the sentinel service and it will return master and read replicas IP just like API response, then connect to specific.
Inside application code you can use as per requirement now as you are getting both write and read IP with sentinel.
Inside the helm chart you can see one configuration
sentinel:
enabled: true
In the bitnami docs they explicitly mention This command will return the address of the current master, which can be accessed from inside the cluster..
Read more at : https://github.com/bitnami/charts/tree/master/bitnami/redis#master-replicas-with-sentinel