Introduction to TestNG

1. Overview

In this article, we’ll introduce the TestNG testing framework.

We’ll focus on: framework setup, writing simple test case and configuration, test execution, test reports generation, and concurrent test execution.

2. Setup

Let’s start by adding the Maven dependency in our pom.xml file:

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>6.11</version>
    <scope>test</scope>
</dependency>

The latest version can be found in the Maven repository.

When using Eclipse, the TestNG plugin may be downloaded and installed from the Eclipse Marketplace.

3. Writing a Test Case

To write a test using TestNG, we just need to annotate the test method with org.testng.annotations.Test annotation:

@Test
public void givenNumber_whenEven_thenTrue() {
    assertTrue(number % 2 == 0);
}

4. Test Configurations

While writing test cases, often we need to execute some configuration or initialization instructions before test executions, and also some cleanup after completion of tests. TestNG provides a number of initialization and clean-up features at method, class, group and suite levels:

@BeforeClass
public void setup() {
    number = 12;
}

@AfterClass
public void tearDown() {
    number = 0;
}

The setup() method annotated with @BeforeClass annotations will be invoked before execution of any methods of that test class, and tearDown() after execution all methods of the test class.

Similarly, we can use the @BeforeMethod, @AfterMethod, @Before/AfterGroup, @Before/AfterTest and @Before/AfterSuite annotations for any configuration at method, group, test and suite levels.

5. Test Execution

We can run the test cases with Maven’s “test” command, it will execute all the test cases annotated with @Test putting them to a default test suite. We can also run test cases from the TestNG test suite XML files, by using the maven-surefire-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>
               src\test\resources\test_suite.xml
            </suiteXmlFile>
        </suiteXmlFiles>
    </configuration>
</plugin>

Note that if we have multiple XML files, covering all test cases, we can add all of them in the suiteXmlFiles tag:

<suiteXmlFiles>
    <suiteXmlFile>
      src/test/resources/parametrized_test.xml
    </suiteXmlFile>
    <suiteXmlFile>
      src/test/resources/registration_test.xml
    </suiteXmlFile>
</suiteXmlFiles>

In order to run the test standalone, we need to have the TestNG library in the classpath and the compiled test class along with the XML configuration file:

java org.testng.TestNG test_suite.xml

6. Grouping Tests

Tests can be run in groups, for example out of 50 test cases 15 can be grouped together and executed leaving others as is.

In TestNG grouping tests in suites are done using XML file:

<suite name="suite">
    <test name="test suite">
        <classes>
            <class name="com.baeldung.RegistrationTest" />
            <class name="com.baeldung.SignInTest" />
        </classes>
    </test>
</suite>

Notice that, both the test classes RegistrationTest, SignInTest now belongs to the same suite and once suite is executed, test cases in this class will get executed.

Apart from test suites, we can also create test groups in TestNG, where instead of test classes methods are grouped together. In order to do that, add the groups parameter in the @Test annotation:

@Test(groups = "regression")
public void givenNegativeNumber_sumLessthanZero_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
 
    assertTrue(sum < 0);
}

Let’s use an XML to execute the groups:

<test name="test groups">
    <groups>
        <run>
            <include name="regression" />
        </run>
    </groups>
    <classes>
        <class
          name="com.baeldung.SummationServiceTest" />
    </classes>
</test>

This will execute the test method tagged with group regression,  in the SummationServiceTest class.

7. Parameterized Tests

Parameterized unit tests are used for testing the same code under several conditions. With the help of parameterized unit tests, we can set up a test method that obtains data from some data source. The main idea is to make the unit test method reusable and to test with a different set of inputs.

In TestNG, we can parametrize tests using @Parameter or @DataProvider annotation. While using the XML file annotate the test method with @Parameter:

@Test
@Parameters({"value", "isEven"})
public void
  givenNumberFromXML_ifEvenCheckOK_thenCorrect(int value, boolean isEven) {
    
    assertEquals(isEven, value % 2 == 0);
}

<suite name="My test suite">
    <test name="numbersXML">
        <parameter name="value" value="1"/>
        <parameter name="isEven" value="false"/>
        <classes>
            <class name="baeldung.com.ParametrizedTests"/>
        </classes>
    </test>
</suite>

@DataProvider(name = "numbers")
public static Object[][] evenNumbers() {
    return new Object[][]{{1, false}, {2, true}, {4, true}};
}
 
@Test(dataProvider = "numbers")
public void 
  givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect(Integer number, boolean expected) {    
    assertEquals(expected, number % 2 == 0);
}

@Test(dataProvider = "numbersObject")
public void 
  givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect(EvenNumber number) {  
    assertEquals(number.isEven(), number.getValue() % 2 == 0);
}
 
@DataProvider(name = "numbersObject")
public Object[][] parameterProvider() {
    return new Object[][]{{new EvenNumber(1, false)},
      {new EvenNumber(2, true)}, {new EvenNumber(4, true)}};
}

@Test(enabled=false)
public void givenNumbers_sumEquals_thenCorrect() { 
    int sum = numbers.stream.reduce(0, Integer::sum);
    assertEquals(6, sum);
}

@Test
public void givenEmail_ifValid_thenTrue() {
    boolean valid = email.contains("@");
 
    assertEquals(valid, true);
}
 
@Test(dependsOnMethods = {"givenEmail_ifValid_thenTrue"})
public void givenValidEmail_whenLoggedIn_thenTrue() {
    LOGGER.info("Email {} valid >> logging in", email);
}

10. Concurrent Test Execution

TestNG allows tests to run in parallel or in multi-threaded mode, thus providing a way to test these multi-threaded pieces of code.

You can configure, for methods, classes, and suites to run in their own threads reducing the total execution time.

10.1. Classes and Methods in Parallel

To run test classes in parallel, mention the parallel  attribute in the suite tag in XML configuration file, with value classes:

<suite name="suite" parallel="classes" thread-count="2">
    <test name="test suite">
        <classes>
	    <class name="baeldung.com.RegistrationTest" />
            <class name="baeldung.com.SignInTest" />
        </classes>
    </test>
</suite>

Note that, if we have multiple test tags in the XML file, these tests can also be run in parallel, by mentioning parallel =” tests”.  Also to execute individual methods in parallel, mention parallel =” methods”.

10.2. Multi-Threaded Execution of Test Method

Let’s say we need to test the behavior of a code when running in multiple threads. TestNG allows to run a test method in multiple threads:

public class MultiThreadedTests {
    
    @Test(threadPoolSize = 5, invocationCount = 10, timeOut = 1000)
    public void givenMethod_whenRunInThreads_thenCorrect() {
        int count = Thread.activeCount();
 
        assertTrue(count > 1);
    }
}

The threadPoolSize indicates that the method will run in n number of threads as mentioned. The invocationCount and timeOut indicate that the test will be executed multiple times and fail the test if it takes more time.

11. Functional Testing

TestNG comes with features which can be used for functional testing as well. In conjunction with Selenium, it can either be used to test functionalities of a web application or used for testing web services with HttpClient.

More details about functional testing with Selenium and TestNG is available here. Also some more pieces of stuff on integration testing in this article.

12. Conclusion

In this article, we had a quick look at how to setup TestNG and execute a simple test case, generate reports, concurrent execution of test cases and also a little about functional programming. For more features like dependent tests, ignoring test cases, test groups and suites you can refer our JUnit vs TestNG article here.

The implementation of all the code snippets can be found over on Github.