I am running tests from two environments:
Jenkins - Ubuntu - no X`s - always the same tests fail. Windows 8 (from localhost) - all tests are doing fine.
Tests are send and executed on a different (from the 2 above) remote selenium server which is seton win 2k8 serv. Any ideas to why those test would fail when running from ubuntu and not from windows ?
Using TestNG, parallel - false, max-thread - tried from 1 to many in suite cfg. Tests are running on chrome (most stable imho).
Actually this is no strange situation. Since in our team we also use Jenkins + TestNG and such cases are not impossible. We also used to rebuild from VCS (Jenkins config) for each run and environment. But I prefer to use the very first of the Continuous delivery's Best practices
Only Build Your Binaries Once
The concept strongly recommend usage of binary files instead of Version control system. Even more - it's antipattern to promote at the source-code level rather than at the binary level. Binaries that get deployed into different environments should be exactly the same as those that went out from the DEV stage, this is checked by storing hashes of the binaries at the time they are created and verifying that the binary is identical at every subsequent stage in the process. This forces you to separate code, which remains the same between environments,and configuration, which differs between environments. You should aim to use the same application binaries throughout your deployment process. Avoid rebuilding from source or processing binaries in any way, even if you believe it is safe to do so.