A test which runs fine locally fails on gitlab.com's GitLab CI shared runners (the MCVE is at https://gitlab.com/krichter/gitlab-ci-file-creation, the log at https://gitlab.com/krichter/gitlab-ci-file-creation/-/jobs/40064276) because during the extraction of the Arquillian webdriver a directory cannot be created or is considered inexisting by Java (see details below):
Nov 13, 2017 7:31:32 AM org.glassfish.deployment.admin.DeployCommand execute
INFO: 664320b4-384a-414a-b0b4-546937428842 was successfully deployed in 7,496 milliseconds.
Nov 13, 2017 7:31:33 AM org.arquillian.spacelift.Spacelift$SpaceliftInstance <init>
INFO: Initialized Spacelift from defaults, workspace: /builds/krichter/gitlab-ci-file-creation, cache: /root/.spacelift/cache
Drone: downloading phantomjs-2.1.1-linux-x86_64.tar.bz2 from https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 to /root/.arquillian/drone/phantomjs/2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
................
Nov 13, 2017 7:31:34 AM org.jboss.arquillian.core.impl.ObserverImpl resolveArguments
WARNING: Argument 1 for UpdateTestResultBeforeAfter.update is null. It won't be invoked.
Nov 13, 2017 7:31:34 AM org.jboss.arquillian.core.impl.ObserverImpl resolveArguments
WARNING: Argument 1 for UpdateTestResultBeforeAfter.update is null. It won't be invoked.
Nov 13, 2017 7:31:34 AM org.jboss.arquillian.core.impl.ObserverImpl resolveArguments
WARNING: Argument 1 for ReusableRemoteWebDriverExtension.destroyLastRemoteWebDriver is null. It won't be invoked.
Nov 13, 2017 7:31:35 AM com.sun.enterprise.admin.cli.embeddable.DeployerImpl undeploy
INFO: 664320b4-384a-414a-b0b4-546937428842 was successfully undeployed
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 19.022 sec <<< FAILURE! - in richtercloud.gitlab.ci.file.creation.GitLabCIIT
testSomething(richtercloud.gitlab.ci.file.creation.GitLabCIIT) Time elapsed: 2.195 sec <<< ERROR!
java.lang.IllegalStateException: Something bad happened when Drone was trying to download and prepare a binary. For more information see the cause.
at org.jboss.arquillian.drone.webdriver.binary.handler.AbstractBinaryHandler.checkAndSetBinary(AbstractBinaryHandler.java:62)
at org.jboss.arquillian.drone.webdriver.factory.PhantomJSDriverFactory.getCapabilities(PhantomJSDriverFactory.java:99)
at org.jboss.arquillian.drone.webdriver.factory.PhantomJSDriverFactory.createInstance(PhantomJSDriverFactory.java:72)
at org.jboss.arquillian.drone.webdriver.factory.PhantomJSDriverFactory.createInstance(PhantomJSDriverFactory.java:37)
at org.jboss.arquillian.drone.webdriver.factory.WebDriverFactory.createInstance(WebDriverFactory.java:127)
at org.jboss.arquillian.drone.webdriver.factory.WebDriverFactory.createInstance(WebDriverFactory.java:38)
at org.jboss.arquillian.drone.impl.DroneConfigurator$1.createInstance(DroneConfigurator.java:112)
at org.jboss.arquillian.drone.impl.CachingCallableImpl.call(CachingCallableImpl.java:44)
at org.jboss.arquillian.core.impl.threading.ThreadedExecutorService$ContextualCallable.call(ThreadedExecutorService.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: The phantomJS binary is not present on the expected path target/drone/1c947d57fce2f21ce0b43fe2ed7cd361/phantomjs-2.1.1-linux-x86_64/bin/phantomjs
at org.jboss.arquillian.drone.webdriver.binary.handler.PhantomJSDriverBinaryHandler.prepare(PhantomJSDriverBinaryHandler.java:56)
at org.jboss.arquillian.drone.webdriver.binary.handler.AbstractBinaryHandler.downloadAndPrepare(AbstractBinaryHandler.java:226)
at org.jboss.arquillian.drone.webdriver.binary.handler.AbstractBinaryHandler.downloadAndPrepare(AbstractBinaryHandler.java:208)
at org.jboss.arquillian.drone.webdriver.binary.handler.AbstractBinaryHandler.downloadAndPrepare(AbstractBinaryHandler.java:175)
at org.jboss.arquillian.drone.webdriver.binary.handler.AbstractBinaryHandler.checkAndSetBinary(AbstractBinaryHandler.java:60)
... 12 more
Nov 13, 2017 7:31:36 AM org.jboss.arquillian.core.impl.ObserverImpl resolveArguments
WARNING: Argument 1 for SeleniumServerExecutor.stopSeleniumServer is null. It won't be invoked.
Nov 13, 2017 7:31:36 AM org.glassfish.admin.mbeanserver.JMXStartupService shutdown
INFO: JMXStartupService and JMXConnectors have been shut down.
JdbcRuntimeExtension, getAllSystemRAResourcesAndPools = [GlassFishConfigBean.org.glassfish.jdbc.config.JdbcResource, GlassFishConfigBean.org.glassfish.jdbc.config.JdbcResource, GlassFishConfigBean.org.glassfish.jdbc.config.JdbcConnectionPool, GlassFishConfigBean.org.glassfish.jdbc.config.JdbcConnectionPool, GlassFishConfigBean.org.glassfish.jdbc.config.JdbcConnectionPool, GlassFishConfigBean.org.glassfish.jdbc.config.JdbcResource]
Nov 13, 2017 7:31:36 AM com.sun.enterprise.v3.server.AppServerStartup stop
INFO: Shutdown procedure finished
I'm using
<dependencies>
[...]
<dependency>
<groupId>org.jboss.shrinkwrap.descriptors</groupId>
<artifactId>shrinkwrap-descriptors-api-javaee</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.junit</groupId>
<artifactId>arquillian-junit-container</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.shrinkwrap.resolver</groupId>
<artifactId>shrinkwrap-resolver-depchain</artifactId>
<type>pom</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.graphene</groupId>
<artifactId>graphene-webdriver</artifactId>
<version>2.2.0</version>
<type>pom</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>htmlunit-driver</artifactId>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.24</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-firefox-driver</artifactId>
</dependency>
<!-- <dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>-->
<dependency>
<groupId>ru.yandex.qatools.ashot</groupId>
<artifactId>ashot</artifactId>
<version>1.5.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.9.2</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.jboss.arquillian</groupId>
<artifactId>arquillian-bom</artifactId>
<version>1.1.13.Final</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.selenium</groupId>
<artifactId>selenium-bom</artifactId>
<version>3.7.0</version>
<!-- - 3.3.1, 3.2.0 and 3.1.0 cause required: java.util.function.Function<? super org.openqa.selenium.WebDriver,V>
found: org.openqa.selenium.support.ui.ExpectedCondition<org.openqa.selenium.WebElement>
reason: cannot infer type-variable(s) V
(argument mismatch; org.openqa.selenium.support.ui.ExpectedCondition<org.openqa.selenium.WebElement> cannot be converted to java.util.function.Function<? super org.openqa.selenium.WebDriver,V>)
- 3.0.1 causes java.lang.IllegalAccessError: tried to access class org.openqa.selenium.os.ExecutableFinder from class org.openqa.selenium.phantomjs.PhantomJSDriverService when using phantomjs driver
- 3.6.0 causes `Unrecognized platform: linux-unknown-64bit`
which one is supposed to work around using 3.5.3
<ref>https://github.com/SeleniumHQ/selenium/issues/4781</ref>-->
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.extension</groupId>
<artifactId>arquillian-drone-bom</artifactId>
<version>2.1.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<profiles>
<profile>
<id>arquillian-glassfish-embedded</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.jboss.arquillian.container</groupId>
<artifactId>arquillian-glassfish-embedded-3.1</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
<groupId>fish.payara.extras</groupId>
<artifactId>payara-embedded-all</artifactId>
<version>4.1.2.174-SNAPSHOT</version>
<!-- - 4.1.2.173 causes `java.lang.NoClassDefFoundError: fish/payara/nucleus/healthcheck/stuck/StuckThreadsStore`
- 4.1.2.172 causes `Caused by: java.lang.ClassNotFoundException: fish.payara.notification.eventbus.EventbusMessage` -->
<scope>test</scope>
</dependency>
</dependencies>
<build>
<testResources>
<testResource>
<directory>src/test/resources</directory>
</testResource>
<testResource>
<directory>src/test/resources-glassfish-embedded</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
and activated the profile.
I downloaded, extracted and investigated phantomjs-2.1.1-linux-x86_64.tar.bz2 from http://phantomjs.org/download.html and verified that phantomjs-2.1.1-linux-x86_64/examples/colorwheel.js exists inside it. Arquillian Drone Webdriver uses the MD5 sum 1c947d57fce2f21ce0b43fe2ed7cd361 as target directory which matches the one of the downloaded .tar.bz2.
The error is located in Arquillian Spacelift which is used by Arquillian Drone to extract the phantomjs driver and which you can't see from the stacktrace because Spacelift ignores failures of directory creation (see https://github.com/arquillian/arquillian-spacelift/pull/34 for details and a proposed fix which you can use by specifying
- git clone --branch file_utils_mkdir https://github.com/krichter722/arquillian-spacelift.git && cd arquillian-spacelift && mvn --batch-mode install && cd ..
in the main job in .gitlab-ci.yml and changing the Spacelift dependency from 1.0.2 to 1.0.3-SNAPSHOT in the POM.
Changing the Spacelift source in the same way as described above and adding explicit directory creation and touch statements and even using Python from within Java to make sure the directory is created with
int pythonDebugReturncode = Runtime.getRuntime().exec(new String[] {"python3",
"-c",
String.format("'import os;os.makedirs(\"%s\");'",
file.getParentFile().getAbsoluteFile().getAbsolutePath())}).waitFor();
System.out.println("pythonDebugReturncode: "+pythonDebugReturncode);
pythonDebugReturncode = Runtime.getRuntime().exec(new String[] {"python3",
"-c",
String.format("'from pathlib import Path;Path(\"%s\").touch();'",
file.getAbsoluteFile().getAbsolutePath())}).waitFor();
System.out.println("pythonDebugReturncode: "+pythonDebugReturncode);
result in 0s being printed as returncodes, but the directory use with the relative path to fail in Java code.
I tried the docker images ubuntu:xenial, ubuntu:trusty, ubuntu:artful, debian:sid and debian:buster. Afaik only Linux-based systems can be used as docker images. I don't have hope to get more insight by testing more images.
I experience
java.net.UnknownHostException: runner-3bd5f424-project-4630314-concurrent-0: runner-3bd5f424-project-4630314-concurrent-0: Temporary failure in name resolution
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
but I assume it's irrelevant for the issue since network filesystems aren't involved and name resolution during permission gaining isn't used since the build runs as root user.
Arquillian Spacelift or Arquillian Drone Webdriver fail to resolve the working directory which isn't changed if Maven is started in an aggregator project root and the Maven basedir consistently. I reported this at https://issues.jboss.org/browse/ARQ-2154.
This means that one has to split the normal run of mvn install into on run doing the compilation with -DskipTests=true -DskipITs=true and then change the directory to the submodule directory and run mvn verify.