java multithreading maven testng testng-dataprovider

Dataprovider+TestNG - Exception in thread "surefire-forkedjvm-command-thread" java.lang.OutOfMemoryError: Java heap space

I have a test automation framework based on testng and dataprovider . I run my code on server with the help of jenkins job . This job executes testcases using maven command line . This job was taking a time of over 3 hrs to execute so I wanted to run my test cases in parallel/concurrently for which I added parallel=true to my dataprovider method.

@DataProvider(name = "data-provider",**parallel = true**)
    protected Object[][] Credentials { return objArray[][];}

But when this jenkins job is run on Azure then I am getting below exception .

Exception in thread "surefire-forkedjvm-command-thread" java.lang.OutOfMemoryError: Java heap space at java.util.concurrent.ConcurrentLinkedQueue.iterator(ConcurrentLinkedQueue.java:669) at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.insertToListeners(CommandReader.java:469) at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:424) at java.lang.Thread.run(Thread.java:750)

Please let me know what are my options as it seems my heap size gets exhausted . I also tried to specify below in my pom.xml but did not work .

<artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <argLine>-Xms1024m -Xmx2048m</argLine>

No thread-count Or parallel details specified in TestNG.xml & neither in dataprovider method . Just used this @DataProvider(name = "data-provider",parallel = true)

However earlier I tried using below in testng.xml . But may be I was'nt able to configure the threads correctly bcos while running the suite it used to get stuck .

<suite name="API AMS regression suite" verbose="10"  data-provider-thread-count="20" >
  
    <test name="POS Testcases">
        <classes>
            <class name="com.qa.apis.tests.Pos"/>
        </classes>
    </test>
</suites>

Solution

The root cause for the OutOfMemoryError is most likely due to your tests creating a lot of threads.

TestNG has 2 thread pools at it's disposal

thread-count - Represents how many regular test methods (which are NOT driven by a data provider) can run in parallel. It has a default value of 5.
data-provider-thread-count - Represents how many data driven tests can run in parallel at any given point in time. It also requires that you set @DataProvider(parallel=true). It has a default value of 10.

Let's assume that you set parallel=methods at the <suite> level or at the <test> level :

Let's say you have 5 test methods in your project then TestNG will run all the 5 methods in parallel.
Now, if all your 5 test methods are actually data provider powered methods and configured to run in parallel via @DataProvider(parallel=true) then TestNG is essentially going to run 5*10 = 50 threads in parallel.

Depending on your machine configuration and your memory configuration, the OS may or may not be able to create that many number of threads.

You have a few options to bring this to your control.

Reduce the number of threads for data-provider-thread-count and also explicitly specify a value for thread-count
Move to using TestNG 7.10.2 which now contains a couple of new features that will help you manage this by allowing you to use a shared thread pool.

For the sake of completeness, I am quoting the relevant sections from the documentation

share-thread-pool-for-data-providers - When this attribute is set to true at the suite level, TestNG will start using a shared thread pool for all the data driven tests in a given . The size of the thread pool is determined using the attribute data-provider-thread-count. This attribute has a default value of false.

use-global-thread-pool - When this attribute is set to true at the suite level, TestNG will start using a common thread pool for running both your regular test methods and data driven test methods. The size of the thread pool is determined using the attribute thread-count. This attribute has a default value of false.

For a detailed explanation please see https://testng.org/#_controlling_threadpool_usage

PS: When you use use-global-thread-pool and when you have some data driven tests, there's a chance that you may stumble into this bug https://github.com/testng-team/testng/issues/3028 Please go through this defect to understand what causes this problem.

This issue has been fixed in the latest snapshot version of TestNG and should be available for use with 7.11.0. Basically this new version will attempt at detecting possible deadlocks and warn you accordingly by halting your execution.