junit5cucumber-javacucumber-junit

Configure parallelism on features and scenarios level in the same time - cucumber with Junit5

I checked some similar questions, but did not found an answer for mine. The most similar question was this question, but it's for TestNG. May be somebody faced an issue like me and solved it already?

I use cucumber with junit.

I have several feature files. Each feature file is designed to test one application instance: A.feature, B.feature, C.feature. Each feature file contains one scenario template(outline) with N examples (usually more than 10). Application instances are independet from each other. And each application instance can process input data in M threads, usually 2. So it does not make sense to run scenarios for particular application instance in more than M (2) threads, because it will lead to flaky tests and unstable execution time. In the same time running scenarios for one instance (with execution-mode.feature=same_thread) in one thread can take much time.

What do I want to achieve:

Example:

Being executed simultaneusly:

A.feature
- scenario 1
- scenario 2

B.feature
- scenario 1
- scenario 2

C.feature
- scenario 1
- scenario 2

Waiting in the queue:

A.feature
- scenario 3
- scenario 4
...
- scenario 10

B.feature
- scenario 3
- scenario 4
...
- scenario 10

C.feature
- scenario 3
- scenario 4
...
- scenario 10

Graphical visualisation for better understanding:

enter image description here

I suppose that it may be achieved using CustomStrategy: https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit-platform-engine#parallel-execution

To control properties such as the desired parallelism and maximum parallelism, Cucumber supports JUnit 5s ParallelExecutionConfigurationStrategy. Cucumber provides two implementations: dynamic and fixed that can be set through cucumber.execution.parallel.config.strategy. You may also implement a custom strategy.

But it could be tricky especially with the fact that I do not understand for now, at what point of time these properties are being set. And could they be changed dynamically (another words based on feature files configuration, not only pom.xml).

My idea was to try using tags on scenario level, something like @feature=A, @feature=B etc to limit parallelism for pool of scenarios. But I'm on the beginning of my journey, so need an advice if it's possible or not.

UPDATE As mentioned in the accepted answer, current implementation of Junit5 does not allow us to do it. Raised a feature request: https://github.com/junit-team/junit5/issues/3524

Solution

  • JUnit 5 handles parallel execution for Cucumber. What you are looking to do is currently not possible. While JUnit 5 does support exclusive resources, it does not support resource pools. You would have to ask the JUnit 5 team if they would support this and possibly provide a pull request.

    But you could implement this in your own code with something like https://commons.apache.org/proper/commons-pool/

    Then for each system you're testing against, the pool should contain 2 tokens.

    @Resource1
Feature: Example 1

  Scenario Outline: Examples

@Resource2
Feature: Example 2

  Scenario Outline: Examples

    Then before starting the scenario you block and acquire the resources from the pool.

        
    static ObjectPool<?> pool = ...

    @Before
    public void before(Scenario scenario) {
        var tags = scenario.getTags(); // Will contain either @Resource1 or @Resource2.

        // block and acquire a resource from the pool
    }

    @After
    public void after(Scenario scenario) {
        var tags = scenario.getTags();

        // return a resource to the the pool
    }

    You would have quite a few threads waiting to acquire a resource from the pool, and definitely have more threads then CPUs, which is not pretty, but I do reckon it is manageable.

    Note: Haven't used this myself.