I can't get a basic Spring Boot Application with Testcontainers being built using Nix, because of a build time dependency on a working docker environment. I put my efforts into my GitHub in order to preserve my learnings from now on.
linux-builder = {
enable = true;
ephemeral = true;
maxJobs = 4;
config = {
nix.settings.sandbox = false; # cannot get it working in a pure fashion
networking = {
nameservers = [ "8.8.8.8" "1.1.1.1" ];
};
virtualisation = {
darwin-builder = {
diskSize = 40 * 1024;
memorySize = 8 * 1024;
};
docker = {
enable = true;
rootless = {
enable = true;
setSocketVariable = true;
};
};
cores = 6;
};
};
};
My Flake looks like that:
{
description = "Inventory Backend Flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
version = "0.0.1-SNAPSHOT";
inventory-jre = pkgs.stdenv.mkDerivation {
name = "inventory-jre";
buildInputs = [ pkgs.openjdk17 ];
src = self;
buildPhase = ''
jlink --add-modules java.base,java.xml --output custom-jre
'';
installPhase = ''
mkdir -p $out
cp -r custom-jre/* $out/
chmod +x $out/bin/*
'';
};
application = pkgs.stdenv.mkDerivation {
# disabling sandbox
__noChroot = true;
name = "inventory-backend";
src = self;
version = version;
buildInputs = [ pkgs.openjdk17 ];
buildPhase = ''
export GRADLE_USER_HOME=$(mktemp -d)
chmod +x ./gradlew
./gradlew clean build --info
'';
installPhase = ''
mkdir -p $out
cp -r build/libs/inventory-backend-${version}.jar $out/
'';
};
dockerImage = pkgs.dockerTools.buildImage {
name = "inventory-backend";
tag = "latest";
created = builtins.substring 0 8 self.lastModifiedDate;
copyToRoot = [application inventory-jre];
config = {
Cmd = [ "${inventory-jre}/bin/java" "-jar" "${application}/inventory-${version}.jar" ];
ExposedPorts = {
"8080/tcp" = {};
};
Volumes = {
"/tmp" = {};
};
};
};
in {
devShells.default = pkgs.mkShell {
buildInputs = [ pkgs.openjdk17 ];
};
packages.default = application;
packages.dockerImage = dockerImage;
defaultPackage = self.packages.default;
}
);
}
In order to build the docker image, I'm running nix build -vvv .#packages.aarch64-linux.dockerImage --print-out-paths, because I want to create a docker image to be run on aarch64-linux docker.
Trying to build the image as described above, it actually fails, because Testcontainers wouldn't find a valid Docker environment when building for aarch64.
Even adding a working docker environment to my linux-builder did not do the trick.
Error log:
2024-03-18T14:39:38.826Z ERROR 3789 --- [ Test worker] o.t.d.DockerClientProviderStrategy : Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
As no valid configuration was found, execution cannot continue.
See https://www.testcontainers.org/on_failure.html for more details.
2024-03-18T14:39:39.831Z ERROR 3789 --- [ Test worker] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:256)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:247)
at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:150)
at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:186)
at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:104)
at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:108)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:321)
at org.testcontainers.jdbc.ContainerDatabaseDriver.connect(ContainerDatabaseDriver.java:134)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
DOCKER_HOSTMy linux-builder has a valid DOCKER_HOST and a working docker setup:
[builder@nixos:~]$ echo $DOCKER_HOST
unix:///run/user/1000/docker.sock
However in the log it says WARN 3789 --- [ Test worker] o.t.d.DockerClientProviderStrategy : DOCKER_HOST unix:///var/run/docker.sock is not listening.
So I suppose, it uses a separate environment.
How do I set up a docker build dependency in Nix(OS) correctly?
Nix Flakes do provide the possibility to add tests by putting derivations in the checks attribute and running them as nix flake check.
Therefore I skip Gradle tests in the build process and instead run them as checks, which seems the more correct way to fix that issue.
It's probably not perfect yet (I would appreciate contribution), but my Flake looks like that now:
{
description = "Inventory Backend Flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
version = "0.0.1-SNAPSHOT";
inventory-jre = pkgs.stdenv.mkDerivation {
name = "inventory-jre";
buildInputs = [ pkgs.openjdk17 ];
src = self;
buildPhase = ''
jlink --add-modules java.base,java.xml --output custom-jre
'';
installPhase = ''
mkdir -p $out
cp -r custom-jre/* $out/
chmod +x $out/bin/*
'';
};
applicationSource = pkgs.stdenv.mkDerivation {
name = "inventory-backend-src";
src = self;
version = version;
installPhase = ''
mkdir -p $out
cp -r ./* $out/
'';
};
application = pkgs.stdenv.mkDerivation {
# disabling sandbox
__noChroot = true;
name = "inventory-backend";
version = version;
buildInputs = [ pkgs.openjdk17 ];
buildPhase = ''
export GRADLE_USER_HOME=$(mktemp -d)
chmod +x ./gradlew
./gradlew clean build --info
'';
installPhase = ''
mkdir -p $out
cp -r build/libs/inventory-backend-${version}.jar $out/
'';
};
dockerImage = pkgs.dockerTools.buildImage {
name = "inventory-backend";
tag = "latest";
created = builtins.substring 0 8 self.lastModifiedDate;
copyToRoot = [application inventory-jre];
config = {
Cmd = [ "${inventory-jre}/bin/java" "-jar" "${application}/inventory-${version}.jar" ];
ExposedPorts = {
"8080/tcp" = {};
};
Volumes = {
"/tmp" = {};
};
};
};
in {
devShells.default = pkgs.mkShell {
buildInputs = [ pkgs.openjdk17 ];
};
packages.default = application;
packages.dockerImage = dockerImage;
checks.gradletests = pkgs.testers.runNixOSTest {
name = "Gradle Test: Inventory Backend Stub";
nodes = {
machine1 = { pkgs, ... }: {
environment.systemPackages = [pkgs.openjdk17 applicationSource];
nix.settings.sandbox = false;
virtualisation.docker.enable = true;
virtualisation.memorySize = 2 * 1024;
virtualisation.msize = 128 * 1024;
virtualisation.cores = 2;
};
};
testScript = ''
machine1.wait_for_unit("network-online.target")
machine1.execute("cp -r ${applicationSource}/* ${applicationSource}/.* .")
machine1.execute("java -version")
machine1.succeed("./gradlew test --no-daemon --debug");
'';
};
}
);
}
Close to the bottom you find the test definition in the attribute checks.gradletests. It essentially just runs ./gradlew test --no-daemon --debug in a NixOS-VM run by qemu-kvm. Because I can define the NixOS-VM essentially just like every NixOS-Host, I can simply enable docker with a one-liner.
Also check out the reference for NixOS Integration Tests and perhaps helpful blog posts like this one.