I am working on a Quarkus app that uses the smallrye microprofile fault tolerance implementation.
We have configured fault tolerance on the client definitions via the annotations API (@Retry, @Bulkhead, etc) and it seems to work but we don't get any sort of feedback about what is happening. Ideally we would like to get some sort of callback but even just having logs would help out in the first step.
The rest clients look something like this:
@RegisterRestClient(configKey = "foo-backend")
@Path("/backend")
interface FooClient {
@POST
@Retry(maxRetries = 4, delay = 900)
@ExponentialBackoff
@Timeout(value = 3000)
fun getUser(payload: GetFooUserRequest): GetFooUserResponse
}
Looking at the logs, even though we trace all communication, I cannot see any event even if I manually stop foo-backend and start it again before the retires run out.
Our logging config looks like this right now but still nothing
quarkus.rest-client.logging.scope=request-response
quarkus.rest-client.logging.body-limit=2048
quarkus.log.category."org.jboss.resteasy.reactive.client.logging".level=DEBUG
Is there a way to get callbacks when a fault tolerance event happens? Or a setting which logs them out? I also would be interested in knowing when out Circuit Breakers are triggered or when a Bulkhead fills up. Logging them would be good enough for now but Ideally I would like to somehow listen for them.
You can enable DEBUG logging for the io.smallrye.faulttolerance category, and you should get all the information you need.
Specifically for circuit breakers, you can register state change listeners for circuit breakers that have been given a name using @CircuitBreakerName -- just inject CircuitBreakerMaintenance and use onStateChange. See https://smallrye.io/docs/smallrye-fault-tolerance/5.6.0/usage/extra.html#_circuit_breaker_maintenance
There's unfortunately nothing similar for bulkheads yet.