I'm developing a Spring Boot application that uses a global database and a local database (one DB, two schemas). Here’s the current database configuration that works fine with the application (I only send the local config, the global is the same, but obviously with different packages and names):
@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(
basePackages = "com.myapp.local",
entityManagerFactoryRef = "localEntityManagerFactory",
transactionManagerRef = "localTransactionManager"
)
public class LocalDbConfig {
@Value("${spring.datasource.local.url}")
private String url;
@Value("${spring.datasource.local.username}")
private String username;
@Value("${spring.datasource.local.password}")
private String password;
@Value("${spring.datasource.local.driver-class-name}")
private String driverClassName;
@Primary
@Bean(name = "localDbDataSource")
public DataSource localDbDataSource() {
return DataSourceBuilder.create()
.url(url)
.username(username)
.password(password)
.driverClassName(driverClassName)
.build();
}
@Primary
@Bean(name = "localEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean localEntityManagerFactory(EntityManagerFactoryBuilder builder, @Qualifier("localDbDataSource") DataSource localDataSource) {
Map<String, String> props = new HashMap<>();
props.put("hibernate.physical_naming_strategy", CamelCaseToUnderscoresNamingStrategy.class.getName());
return builder
.dataSource(localDataSource)
.packages("com.myapp.local")
.properties(props)
.build();
}
@Primary
@Bean(name = "localTransactionManager")
public PlatformTransactionManager localTransactionManager(@Qualifier("localEntityManagerFactory") EntityManagerFactory localEntityManagerFactory) {
return new JpaTransactionManager(localEntityManagerFactory);
}
}
I want to test the application using two MariaDB containers that start successfully. However, both containers stop with the following error: "Waiting for database connection to become available at jdbc:mysql://x.x.x.x:xxxx/test using query 'SELECT 1"
This is my TestContainersInitializer:
@TestConfiguration
public class TestContainersInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final Network SHARED_NETWORK = Network.newNetwork();
private static final MariaDBContainer<?> localMariaDB = new MariaDBContainer<>(DockerImageName.parse("mariadb:latest"))
.withNetwork(SHARED_NETWORK)
.withDatabaseName("local_db")
.withUsername("root")
.withPassword("test")
.withReuse(true);
private static final MariaDBContainer<?> globalMariaDB = new MariaDBContainer<>(DockerImageName.parse("mariadb:latest"))
.withNetwork(SHARED_NETWORK)
.withDatabaseName("global_db")
.withUsername("root")
.withPassword("test")
.withReuse(true);
private static final KeycloakContainer keycloak = new KeycloakContainer()
.withNetwork(SHARED_NETWORK)
.withRealmImportFile("test-realm-export.json")
.withAdminUsername("keycloakadmin")
.withAdminPassword("keycloakadmin")
.withReuse(true);
private static final KafkaContainer kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"))
.withNetwork(SHARED_NETWORK)
.withReuse(true);
static {
Startables.deepStart(localMariaDB, globalMariaDB, keycloak, kafka).join();
}
@Override
public void initialize(@NotNull ConfigurableApplicationContext applicationContext) {
TestPropertyValues.of(
"spring.datasource.local.url=" + localMariaDB.getJdbcUrl(),
"spring.datasource.local.username=" + localMariaDB.getUsername(),
"spring.datasource.local.password=" + localMariaDB.getPassword(),
"spring.datasource.local.driver-class-name=" + localMariaDB.getDriverClassName(),
"spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MariaDBDialect",
"spring.datasource.global.url=" + globalMariaDB.getJdbcUrl(),
"spring.datasource.global.username=" + globalMariaDB.getUsername(),
"spring.datasource.global.password=" + globalMariaDB.getPassword(),
"spring.datasource.global.driver-class-name=" + globalMariaDB.getDriverClassName(),
"spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MariaDBDialect",
"keycloak.server-url=http://localhost:" + keycloak.getFirstMappedPort(),
"spring.security.oauth2.resourceserver.jwt.issuer-uri=http://localhost:" + keycloak.getFirstMappedPort() + "/realms/app",
"spring.kafka.bootstrap-servers=" + kafka.getBootstrapServers()
).applyTo(applicationContext.getEnvironment());
}
}
I use this in my IntegrationTestBase class in this way:
@ContextConfiguration(initializers = TestContainersInitializer.class)
I don't know what else would be needed for this approach to work (to use the existing configuration with the container data).
I also tried writing a separate test configuration for the databases:
@TestConfiguration
public class TestDatabaseConfiguration {
@Primary
@Bean
public DataSource localDataSource(Environment env) {
return DataSourceBuilder.create()
.url(env.getProperty("spring.datasource.local.url"))
.username(env.getProperty("spring.datasource.local.username"))
.password(env.getProperty("spring.datasource.local.password"))
.driverClassName(env.getProperty("spring.datasource.local.driver-class-name"))
.build();
}
@Bean
@Qualifier("globalDataSource")
public DataSource globalDataSource(Environment env) {
return DataSourceBuilder.create()
.url(env.getProperty("spring.datasource.global.url"))
.username(env.getProperty("spring.datasource.global.username"))
.password(env.getProperty("spring.datasource.global.password"))
.driverClassName(env.getProperty("spring.datasource.global.driver-class-name"))
.build();
}
@Primary
@Bean
public LocalContainerEntityManagerFactoryBean localEntityManagerFactory(
EntityManagerFactoryBuilder builder,
DataSource localDataSource) {
Map<String, String> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto", "validate");
properties.put("hibernate.dialect", "org.hibernate.dialect.MariaDBDialect");
return builder
.dataSource(localDataSource)
.packages("com.utitech.bidhubbackend.local", "com.utitech.bidhubbackend.common.fileupload")
.properties(properties)
.build();
}
@Bean
public LocalContainerEntityManagerFactoryBean globalEntityManagerFactory(
EntityManagerFactoryBuilder builder,
@Qualifier("globalDataSource") DataSource globalDataSource) {
Map<String, String> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto", "validate");
properties.put("hibernate.dialect", "org.hibernate.dialect.MariaDBDialect");
return builder
.dataSource(globalDataSource)
.packages("com.utitech.bidhubbackend.global", "com.utitech.bidhubbackend.common.fileupload")
.properties(properties)
.build();
}
@Primary
@Bean
public PlatformTransactionManager localTransactionManager(
@Qualifier("localEntityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
@Bean
public PlatformTransactionManager globalTransactionManager(
@Qualifier("globalEntityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}
For this approach, in the IntegrationTestBase I used the property properties = {"spring.main.allow-bean-definition-overriding=true"}, which should allow similar-named beans to be overridden. However, regardless of what I try, I continue to face the "waiting for database" problem.
What steps should I take to resolve this issue and successfully test the application with multiple MariaDB containers? Should I modify the existing configuration, or is there a more suitable approach to handle the database connection for testing?
Meanwhile I solved it, more than likely the problem was caused by the test databases not being created correctly as the configuration was not available. Here is the modified code and some explanations:
I used the existing db configuration, which I managed to use properly in the IntegrationTestBase class as follows:
@Import({FlywayConfig.class, LocalDbConfig.class, GlobalDbConfig.class})
I wrote a separate flyway config that provides two separate flyways to ensure correct migration of local and global schemas:
@TestConfiguration
public class FlywayConfig {
@Bean(name = "localFlyway")
public Flyway localFlyway(@Qualifier("localDbDataSource") DataSource localDataSource) {
return Flyway.configure()
.dataSource(localDataSource)
.locations("classpath:db/migration")
.load();
}
@Bean(name = "globalFlyway")
public Flyway globalFlyway(@Qualifier("globalDbDataSource") DataSource globalDataSource) {
return Flyway.configure()
.dataSource(globalDataSource)
.locations("classpath:db/migration/test")
.load();
}
}
I ended up starting the containers not in a separate file (TestContainersInitializer), but in IntegrationTestBase. I changed is the creation of db containers. I created the containers without setting the username and password, as that overwrote the default user and password, which also caused problems.
Here is my IntegrationTestBase class with all the important information:
@SpringBootTest(classes = MyApp.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
@ActiveProfiles("test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@TestPropertySource(locations = "classpath:application-test.properties")
@Import({FlywayConfig.class, LocalDbConfig.class, GlobalDbConfig.class})
public class IntegrationTestBase {
private static final Logger LOGGER = LoggerFactory.getLogger(IntegrationTestBase.class.getName());
@Autowired
public MockMvc mockMvc;
@Autowired
private DatabaseCleaner databaseCleaner;
@Autowired
private @Qualifier("localFlyway") Flyway localFlyway;
@Autowired
private @Qualifier("globalFlyway") Flyway globalFlyway;
@Container
public static final MariaDBContainer<?> localDbContainer = new MariaDBContainer<>("mariadb:latest")
.withUrlParam("serverTimezone", "UTC")
.withReuse(true);
@Container
public static final MariaDBContainer<?> globalDbContainer = new MariaDBContainer<>("mariadb:latest")
.withUrlParam("serverTimezone", "UTC")
.withReuse(true);
static {
Startables.deepStart(localDbContainer, globalDbContainer).join();
}
@DynamicPropertySource
static void setProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.local.url", localDbContainer::getJdbcUrl);
registry.add("spring.datasource.local.username", localDbContainer::getUsername);
registry.add("spring.datasource.local.password", localDbContainer::getPassword);
registry.add("spring.datasource.local.driver-class-name", localDbContainer::getDriverClassName);
registry.add("spring.datasource.global.url", globalDbContainer::getJdbcUrl);
registry.add("spring.datasource.global.username", globalDbContainer::getUsername);
registry.add("spring.datasource.global.password", globalDbContainer::getPassword);
registry.add("spring.datasource.global.driver-class-name", globalDbContainer::getDriverClassName);
}
@BeforeAll
public void setUp() {
LOGGER.info("Running database migrations...");
localFlyway.migrate();
globalFlyway.migrate();
}
In addition, here's my db cleaning logic, which I wrote as a separate component:
@Component
public class DatabaseCleaner {
private static final Logger LOGGER = LoggerFactory.getLogger(DatabaseCleaner.class);
private final DataSource localDataSource;
private final DataSource globalDataSource;
public DatabaseCleaner(@Qualifier("localDbDataSource") DataSource localDataSource,
@Qualifier("globalDbDataSource") DataSource globalDataSource) {
this.localDataSource = localDataSource;
this.globalDataSource = globalDataSource;
}
public void cleanLocal() {
executeCleanupScript(localDataSource, "/clean_up_local.sql");
}
public void cleanGlobal() {
executeCleanupScript(globalDataSource, "/clean_up_global.sql");
}
private void executeCleanupScript(DataSource dataSource, String scriptPath) {
try (Connection connection = dataSource.getConnection()) {
Resource resource = new ClassPathResource(scriptPath);
ScriptUtils.executeSqlScript(connection, resource);
} catch (Exception e) {
LOGGER.error("Error executing cleanup script: " + scriptPath, e);
throw new RuntimeException("Database cleanup failed", e);
}
}
}
I also use it in IntegrationTestBase as follows:
@AfterEach
public void cleanUpEach() {
try {
databaseCleaner.cleanLocal();
databaseCleaner.cleanGlobal();
} catch (Exception e) {
LOGGER.error("Cleanup failed", e);
throw new RuntimeException("Test cleanup failed", e);
}
}