javaautomated-teststestngdataprovidertestng-dataprovider

New data is not passed from TestNG DataProvider to test on test retry

I am using TestNG @DataProvider for passing objects as test parameters and Retry class (that extends RetryAnalyzerCount) to retry failed tests.

All is working fine except: if some of object properties were changed during test before it fails, dataProvider method is running on retry but don't pass new object to test, so in test we have old set data from 1st run.

Example:

    @DataProvider(name = "userData")
    public static Object[] getUserData() {
        System.out.println("DataProvider is started...");
        TestUser testUser = new TestUser("John");
        System.out.println("Username from dataprovider: " + testUser.getName());
        return new Object[]{testUser};
    }

    @Test(retryAnalyzer = Retry.class, dataProvider = "userData")
    public void testRetry(TestUser user) {
        System.out.println("Test is started...");
        System.out.println("Username from test: " + user.getName());
        user.setName("New Name");
        Assert.fail();
    }

Output:

DataProvider is started...
Username from dataprovider: John
Test is started...
Username from test: John

Retry test...

DataProvider is started...
Username from dataprovider: John
Test is started...
Username from test: New Name

TestNG version: 7.6.0, 7.7.1

Facing such issue and don't know how to get around it.

Solution

  • There is no issue here.

    Here's a sample that re-iterates all these aspects. For the sake of completeness I have added the same details to your post on the TestNG google users group as well.

    Test class sample (I am using 7.7.1 which is the latest released version for running this test)

    import com.rationaleemotions.DemoTest.Commentary;
import java.util.concurrent.atomic.AtomicInteger;
import org.testng.Assert;
import org.testng.IExecutionListener;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(Commentary.class)
public class DemoTest {

  public static final AtomicInteger dataProviderInvocationCount = new AtomicInteger(0);

  @Test(dataProvider = "dp", retryAnalyzer = TryAgain.class)
  public void testMethod(Pojo user) {
    System.out.println("Test is started for data " + user.toString() + " with name " + user.getName());
    user.setName("Dragon-Warrior");
    Assert.fail();
  }

  @DataProvider(name = "dp")
  public static Object[][] getTestData() {
    System.err.println(
        "Data Provider invocation count #" + dataProviderInvocationCount.incrementAndGet());
    return new Object[][]{
        {new Pojo().setName("John")}
    };
  }

  public static class TryAgain implements IRetryAnalyzer {

    private int counter = 1;

    @Override
    public boolean retry(ITestResult result) {
      return counter++ != 3;
    }
  }

  public static class Pojo {

    private String name;

    public Pojo() {
      System.err.println("Instantiated " + this);
    }

    public String getName() {
      return name;
    }

    public Pojo setName(String name) {
      this.name = name;
      return this;
    }
  }

  public static class Commentary implements IExecutionListener {

    @Override
    public void onExecutionStart() {}

    @Override
    public void onExecutionFinish() {
      System.err.println(
          "Data Provider was invoked " + dataProviderInvocationCount.get() + " time(s).");
    }
  }
}

    Here's the complete execution output, along with some inline comments that I have added for elaboration:

    Data Provider invocation count #1
Instantiated com.rationaleemotions.DemoTest$Pojo@2bfc268b
Test is started for data com.rationaleemotions.DemoTest$Pojo@2bfc268b with name John

Test ignored.
Data Provider invocation count #2
Instantiated com.rationaleemotions.DemoTest$Pojo@2b91004a
Test is started for data com.rationaleemotions.DemoTest$Pojo@2bfc268b with name Dragon-Warrior

Test ignored.
Data Provider invocation count #3
Instantiated com.rationaleemotions.DemoTest$Pojo@169e6180
Test is started for data com.rationaleemotions.DemoTest$Pojo@2bfc268b with name Dragon-Warrior

java.lang.AssertionError: null

    at org.testng.Assert.fail(Assert.java:110)
    at org.testng.Assert.fail(Assert.java:115)
    at com.rationaleemotions.DemoTest.testMethod(DemoTest.java:22)


===============================================
Default Suite
Total tests run: 3, Passes: 0, Failures: 1, Skips: 0, Retries: 2
===============================================

Data Provider was invoked 3 time(s).

Process finished with exit code 0

    Points to note: