kubernetesredis

what is the redis headless service used for in kubernetes cluster


Today I am facing a problem that the redis in the kubernetes cluster shows error like this:

2021-11-16T05:32:18 [INFO] - increment delete failed...,MobcError(RedisCMDError(An error was signalled by the server: You can't write against a read only replica.))

I check my redis config and found that the config url like this:

redisConnectionStr="redis://default:password@cruise-redis-headless.reddwarf-cache.svc.cluster.local:6379/3"

I check the servcie and found it mapped to the redis cluster master and replica, I was wonder the headness service user for what situation? why not only keep the master servcie and replica servcie? If read operate, use any of the servcie would be fine. If write, only use the master service. when should i use the headness service?


Solution

  • why not only keep the master service and replica servcie?

    You can do it, if you could implement it by your own.

    As redis helm maybe dont use specific labels to run Read/Write replicas so it's hard to divert and create the different services for read & write.

    If you are deploying the Redis using the helm it will create the Two services into the K8s cluster

    1. Headless service

    2. ClusterIP service

    In the your application code side, you should be using the normal clusterIP service.

    So idea is headless return the pod replicas IPs which will be used further by the Redis internally to manage the cluster if you are using the Redis sentinel or Redis cluster.

    If you are using the ClusterIP service it will auto route the traffic to Read/Write replicas based on the condition.

    If you are not using sentinel or cluster simply use ClusterIP and you are good to go.

    Sentinel details

    If you are using the Sentinel feature, you can ping the sentinel service and it will return master and read replicas IP just like API response, then connect to specific.

    Inside application code you can use as per requirement now as you are getting both write and read IP with sentinel.

    Inside the helm chart you can see one configuration

    sentinel:
      enabled: true
    

    In the bitnami docs they explicitly mention This command will return the address of the current master, which can be accessed from inside the cluster..

    Read more at : https://github.com/bitnami/charts/tree/master/bitnami/redis#master-replicas-with-sentinel