How to check if docker Cassandra instance is ready to take connections

I have two docker instances that I launch with docker-compose. One holds a Cassandra instance One holds a Spring Boot application that tries to connect to that application.

However, the Spring Boot application will always fail, because it's trying to connect to a Cassandra instance that is not ready yet to take connections.

I have tried:

• Using restart:always in Docker-compose
  • This still doesn't always work, because the Cassandra might be up 'enough' to no longer crash the Spring Boot application, but not up 'enough' to have successfully created the Table/Column family. On top of that, this is a very hacky solution.
• Using healthcheck
  • It seems like healthcheck in compose doesn't have restart capabilities
• Using a bash script as entrypoint
  • In the hope that I could use netstat,ping,... whatever to determine that readiness state of Cassandra

Right now the only thing that really works is using that same bash script and sleep the process for x seconds, then start the jar. This is even more hacky...

Does anyone have an idea on how to solve this?

Thanks!

Does the spring boot service defined in the docker-compose.yml depends_on the cassandara service? If yes then the service is started only if the cassandra service is ready.

https://docs.docker.com/compose/compose-file/#depends_on

Take a look at this github repository, to find a healthcheck for the cassandra service.

https://github.com/docker-library/healthcheck

CONCLUSION

After some discussion we found out that docker-compose seems not to provide a functionality for waiting until services are up and healthy, such as Kubernetes and Openshift provide (See comments below). They recommend to use wrapper script (docker-entrypoint.sh) which waits for the depending service to come up, which make binaries necessary, the actual service shouldn't use such as the cassandra client binary. Additionally the service depending on cassandra could never get up if cassandra doesn't, which shouldn't happen.

A main thing with microservices is that they have to be resilient for failures and are not supposed to die or not to come up if a depending service is currently not available or unexpectedly disappears. Therefore the microservice should be implemented in a way so that it retries to get connection after startup or an unexpected disappearance. Unexpected is a word actually wrongly used in this context, because you should always expect such issues in a distributed environment, and even with docker-compose you will face issues like that as discussed in this topic.

The following link points to a tutorial which helped to integrate cassandra properly into a spring boot application. It provides a way to implement the retrieval of a cassandra connection with a retry behavior, therefore the service is resilient to a non existing cassandra database and will not fail to start anymore. Hope this helps others as well.

https://dzone.com/articles/containerising-a-spring-data-cassandra-application