I saved all test cases in many excel files. Every sheet in the excel file is a test case. I read all the sheets from a java class as the dataprovider.
I have only 1 test method defined in my test:
@Test(dataProvider = "testcases", dataProviderClass = TestCaseProvider.class)
public void test(TestSheet sheet) {
String name = "File: " + sheet.getFilename() + " -> Sheet: " + sheet.getSheet();
CaseExecutor executor = new CaseExecutor(sheet);
executor.start();
executor.assertPostCondition();
}
Now we have a new requirements that we should execute the tests by excel file one by one. For example:
Excel file A has 3 sheets: SheetA1, SheetA2, SheetA3 Excel file B has 2 sheets: SheetB1, SheetB2
We must ensure the sheets in file A is: SheetA1 -> SheetA2 -> SheetA3, and the order of file B is: SheetB1 -> SheetB2
We don't care the order for different file.
SheetA1 -> SheetB1 -> SheetA2 -> SheetA3 -> SheetB2
SheetA1 -> SheetB1 -> SheetB2 -> SheetA2 -> SheetB3
These 2 orders or other similar orders are all OK for us.
Also we use multi-thread to run our cases.
I think we have 2 solutions to meet the new requirement.
Bind all sheets from a file to 1 execution thread. I can set sheet index as the order.
Set the priority for all sheets. sheet 0 is priority 0, sheet 1 is priority 1.... then we execute all test by the priority. that means we execute the first sheet of all the excel file as the top priority. the 2nd sheet can only execute after all 1st sheet completely.
Does anyone know which solution can be implemented easily? and how to do?
Updates for @Krishnan Mahadevan:
I tried the solution from @Krishnan Mahadevan,
I tried the codes without the suite file. But set the thread count to 5 in my Java code.
Here is my codes:
@BeforeSuite
public void setupClassName(ITestContext context) {
context.getCurrentXmlTest().getSuite().setDataProviderThreadCount(5);
context.getCurrentXmlTest().getSuite().setPreserveOrder(true);
context.getCurrentXmlTest().getSuite().setParallel(XmlSuite.ParallelMode.INSTANCES);
context.getCurrentXmlTest().getSuite().setGroupByInstances(true);
}
And I add a sleep for every case:
@Test(dataProvider = "getAllTestCases")
public void testCase(JsonObject testCase) {
System.err.println("Running test case: " + testCase.toString() + " on thread " + Thread.currentThread().getId());
ThreadUtil.sleep(3, TimeUnit.SECONDS);
}
Then I find that all the test cases are executed one by one, even if they are in different threads.
The output like:
File1 Test1
Pause 3 seconds
File1 Test2
Pause 3 seconds
File2 Test1
Pause 3 seconds
......
My expected output should be:
File1 Test1
File2 Test1
File3 Test1
Pause 3 second
File1 Test2
File2 Test2
File3 Test2
Pause 3 second
......
Do you know what's wrong with the thread count?
You should be able to achieve this if you use a Factory and then power it with a data provider.
Here's a sample that shows this in action.
I am having 2 json files (you can treat them as your .xls files) wherein each json file has a bunch of json objects (you can treat them as individual sheets) to represent multiple test cases. Both the jsons look the same in structure and as below
[
{
"test": "SheetA1"
},
{
"test": "SheetA2"
},
{
"test": "SheetA3"
}
]
package com.rationaleemotions.so.qn78521883;
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.Reader;
import java.nio.file.Files;
import java.nio.file.Path;
public class TestCaseFactory {
private final JsonArray individualTests;
@Factory(dataProvider = "getAllTestFiles")
public TestCaseFactory(JsonArray individualTests) {
this.individualTests = individualTests;
}
@Test(dataProvider = "getAllTestCases")
public void testCase(JsonObject testCase) {
System.err.println("Running test case: " + testCase.toString() + " on thread " + Thread.currentThread().getId());
}
@DataProvider
public Object[][] getAllTestCases() {
int length = individualTests.size();
Object[][] testCases = new Object[length][];
for (int i = 0; i < length; i++) {
testCases[i] = new Object[]{individualTests.get(i)};
}
return testCases;
}
@DataProvider
public static Object[] getAllTestFiles() throws IOException {
Gson gson = new Gson();
return Files.list(Path.of("src/test/resources/jsons"))
.map(TestCaseFactory::reader)
.map(it -> gson.fromJson(it, JsonArray.class))
.toArray(Object[]::new);
}
private static Reader reader(Path path) {
try {
return new FileReader(path.toFile());
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
}
}
}
Now in order for you to leverage parallelism, you would basically do group-by-instances to true so that we have all tests running in the same instance run in the same thread and set the parallelism strategy to instances
Here's the suite file
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="qn_78521883_suite" verbose="2">
<test name="qn_78521883_test" parallel="instances" group-by-instances="true">
<classes>
<class name="com.rationaleemotions.so.qn78521883.TestCaseFactory">
</class>
</classes>
</test>
</suite>
The execution output
...
... TestNG 7.10.2 by Cédric Beust (cedric@beust.com)
...
Running test case: {"test":"SheetA1"} on thread 18
Running test case: {"test":"SheetB1"} on thread 17
Running test case: {"test":"SheetA2"} on thread 18
Running test case: {"test":"SheetB2"} on thread 17
Running test case: {"test":"SheetA3"} on thread 18
Running test case: {"test":"SheetB3"} on thread 17
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetA2"})
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetB1"})
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetB2"})
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetB3"})
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetA1"})
PASSED: com.rationaleemotions.so.qn78521883.TestCaseFactory.testCase({"test":"SheetA3"})
===============================================
qn_78521883_test
Tests run: 2, Failures: 0, Skips: 0
===============================================
===============================================
qn_78521883_suite
Total tests run: 6, Passes: 6, Failures: 0, Skips: 0
===============================================
Process finished with exit code 0