I'm working on a fairly big application, with ~260k LOC and roughly 1800 unit tests on the suite. A little background:
SpringJUnit4ClassRunner
to run the testsforkCount
equals one and reuseForks
equals true). Doing this I am sure I'm using the advantages of Spring's Static Context Cache (reuse the same ApplicationContext
on every test)When we run tests on our development VMs, using Oracle JDK 7u79/80 we see very fast tests. For example, a fairly big test class with 50+ test methods takes something around 1:30m (including context initialisation time) to run.
Our basic Dev VM is something like:
We use Jenkins as our CI server and Maven (3.2) takes care of the build process. We have a Master + 2 Slaves architecture. All the VMs are exactly the same:
Remember those tests that usually take 1:30 min to run on our Eclipse's? Here on the server they take more than 15 minutes to run! Here's what I already tried to do (without success so far.):
Bottom line, is there any concrete reason on why Surefire's execution is s much slower than running JUnit on Eclipse? I'm scratching my head over this for a few days now and it's really starting to annoy me! I hate it when the solution seems to be so close but yet it is so far.
I couldn't test with Maven on my Dev machine because I couldn't allocate all the memory necessary to do it, but running chunks of classes (not the whole 1.8k tests suite) still reveals to be quite slower than running on Eclipse.
I understand Maven has all those phases and so on, but the difference should't be this big. Don't you agree?
Any input will be highly appreciated. I can provide any more information you find necessary!
PS: Surefire v2.17, Maven 3.2.2, JUnit 4.12, Spring Test 3.2.13
Thanks a lot!
UPDATE 1
I tried deactivating JaCoCo on the CI server to see if it affected the build times. It doesn't. The execution times remain the same.
I have made some progress on this. As I said above, this is a multi-modular (20+) Maven application. We keep all our tests separated on one module, but Surefire's configuration were being done on the parent POM, rather on the test module's POM.
After trying everything we decided to bring the test execution configuration to this module alone, skipping all the others. This brought the suite execution from ~45' to 22'. We're scratching our heads over this, and once we understand what is going on I'll post here again.
TL;DR: This image explains everything. (off-topic, safe for work)
Thanks for all your input!