javakubernetesjunit5testcontainersargo-workflows

Testcontainer for Kubernetes and Argo Workflows


I want to do some integration testing + setup local dev environment for a Java/Spring Boot application that interacts with Kubernetes via Argo Workflows. I don't see a Kubernetes test container (probably because K8s is not just one container) but I did notice the K3 container which is supposedly a lightweight distro for kubernetes.

I usually use a locally installed Minikube instance to achieve this but Testcontainers are very cool and I'm already using them for my DB and a Kafka queue, would be nice to have one solution for everything.

Is there a way to use Testcontainers to test an app that interacts with Kubernetes via Argo Wofkflows?


Solution

  • This worked eventually for me.. also see discussion here (https://github.com/spring-projects/spring-boot/issues/45717)

       @Bean
        K3sContainer k3s() {
            K3sContainer k3sContainer = new K3sContainer(DockerImageName.parse("rancher/k3s:v1.21.3-k3s1"))
                    .withCommand("server", "--disable", "metrics-server") // we don't need metrics
                    .withLogConsumer(new Slf4jLogConsumer(logger)
                    );
    
            // don't normally need to start the container in these @Bean methods but can't get the config unless its started
            k3sContainer.start();
    
            String kubeConfigYaml = k3sContainer.getKubeConfigYaml();
            Config config = Config.fromKubeconfig(kubeConfigYaml);  // requires io.fabric8:kubernetes-client:5.11.0 or higher
    
            // in the absence of @ServiceConnection integration for this testcontainer, Jack the test container URL into properties so it's picked up when I create a client in main app
            System.setProperty(Config.KUBERNETES_MASTER_SYSTEM_PROPERTY, config.getMasterUrl());
            System.setProperty(Config.KUBERNETES_CA_CERTIFICATE_DATA_SYSTEM_PROPERTY, config.getCaCertData());
            System.setProperty(Config.KUBERNETES_CLIENT_CERTIFICATE_DATA_SYSTEM_PROPERTY, config.getClientCertData());
            System.setProperty(Config.KUBERNETES_CLIENT_KEY_DATA_SYSTEM_PROPERTY, config.getClientKeyData());
            System.setProperty(Config.KUBERNETES_TRUST_CERT_SYSTEM_PROPERTY, "true");
    
            return k3sContainer;
        }