I have a large set of tests running on a site. When I run the test locally on Windows - they all pass on 100%. The test are designed and running on Google Chrome.
Now, we have started to run the tests on Linux via Jenkins jobs on headless mode. Some tests now fail on 0% or only pass on 20% or even 10%. In my code I'm finding elements by ID, xpath or css and simple click on them. And I use the WebDriverWait object for waiting - both for the element to be present and to be clickable.
Example of my code:
WebDriverWait wait = new WebDriverWait(browser, secondsToWait);
wait.until(ExpectedConditions.presenceOfElementLocated(By.id(elementID)));
lastFoundElement = wait.until(ExpectedConditions.elementToBeClickable(By.id(elementID)));
clickLastFoundElement();
In my report I see mostly that the elements were not found and that I passed the timeout set in the wait object.
How to make headless tests be more stable?
Why the headless state causes so many problems?
It is actually known issue to Selenium community that headless is not stable as it should be, read here about the issue more
Chrome runs very unstable when headless mode is activated.
There are multiple different issues and bugs depending on: Chrome Version, ChromeDriver
Version and the executed tests.
Issues and Fixes (occured so far):
Exception:
No informative Error message. Only a timeout exception when navigate() is called on the driver:
org.openqa.selenium.TimeoutException: timeout
Fix:
options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");
Fix:
options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");
Chrome does not operate correctly after a certain time, when tests run very long resp. many actions are executed in one test session
Exception:
... Timed out receiving message from renderer: ...
Fix (not tested yet!):
options.addArguments("--disable-browser-side-navigation");