We are working on an application using akka-persistence and akka-persistence-jdbc. Every thing works great but lately we had an issue where the database went down for a very short time and we saw persistence failing even when the database was up and we could see the following exception:
akka.pattern.CircuitBreakerOpenException: Circuit Breaker is open; calls are failing fast
Can someome help us with how can we define the circuit breaker configurations ourself?
Since akka-persistence-jdbc doesn't configure its own circuit breakers, it inherits the default Akka Persistence circuit breakers.
The journal's circuit breaker is thus configured at akka.persistence.journal-plugin-fallback.circuit-breaker and the snapshot store's circuit breaker at akka.persistence.snapshot-store-plugin-fallback.circuit-breaker. You'll most likely want to reduce the reset-timeout in both, which is the time from when the breaker trips until it will move to half-open (basically it allows one request through (rejecting everything else) and decides whether to close or open the circuit breaker based on the result). For the journal plugin, the default reset-timeout is 30 seconds and the default for the snapshot store plugin is 60 seconds.