[SOLVED] Is probing of a Pod retried after a readiness probe fails

Is probing of a Pod retried after a readiness probe fails

readinessProbe: Indicates whether the container is ready to respond to requests. If the readiness probe fails, the endpoints controller removes the Pod's IP address from the endpoints of all Services that match the Pod. The default state of readiness before the initial delay is Failure. If a Container does not provide a readiness probe, the default state is Success

If the readiness probe fails (and the Pod's IP address is removed from end point), what happens next? Will the Pod's readiness probe conditions be checked again? Will it check again after initial delay? Is there any chance the Pod's IP address is added to the end point again (if the Pod self healed after readiness probe fails)? Will that Pod receive traffic again incase if it's healed?

Solution

will the pod's readiness prob conditions checked again?

yes, the condition will be checked again depends on the threshold you have set.

On each periodSecondsconfiguration readiness will be checked for POD.

will it check again after the initial delay?

It will check after the Initial delay only. Initial delay comes into the picture when POD is initializing or starting. readiness check will wait for configured time and after that time it will start checking the readiness of POD on each time interval suppose every 5 second or 10 seconds depends on the configuration of the periodSeconds.

are there any chance pod's ip address added to the end point again(if pod self healed after readiness probe fails)?

Yes if get auto healed mean, successThreshold set to 1 time if POD give 200 one time it will marked as heal and running pod is this case POD will get the traffic again.

will the pod ever receive traffic again incase if it's healed?

Yes

For example :

readinessProbe:
            httpGet:
              path: /k8/readiness
              port: 9595
            initialDelaySeconds: 25
            periodSeconds: 8
            timeoutSeconds: 10
            successThreshold: 1
            failureThreshold: 30
        livenessProbe:
            httpGet:
              path: /k8/liveness
              port: 9595
            initialDelaySeconds: 30
            periodSeconds: 8
            timeoutSeconds: 10
            successThreshold: 1
            failureThreshold: 30

The readiness and liveness probe will check the status on HTTP endpoint as mentioned in configuration.

initialDelaySeconds : it will only come into the picture when your POD initializing or starting again due to restart or anything. So when POD is starting readiness wont check service status till 30 second.

After 30 seconds it will try to check the status on the endpoint. If successful POD will be in a ready state to handle the traffic or else it will try for another time periodSeconds so after 8 seconds it will try again if we will 200 response POD will be Ready or else will try after 8 seconds.

timeoutSeconds : single hop or request will wait the amount of time to get response from service or else mark as failed check.

failureThreshold : Maximum number of failed check after this POD will get started or changed to Not Ready state based on configuration liveness or readiness.

successThreshold : Successful threshold means if single request get success response from service POD status gets changed to Ready.

If continuous 30 failureThreshold occur then only POD will get marked as Not ready if in between single successThreshold occur POD will get mark as Ready same for liveness.

Note : Above example is just for reference can not be useful in an actual production scenario.