TestNG Parallel Execution

Master TestNG parallel test execution including thread pool configuration, suite-level parallelism, method-level parallelism, and thread safety patterns. This skill covers techniques for maximizing test throughput while maintaining test reliability.

Overview

TestNG supports parallel execution at multiple levels: suite, test, class, and method. Proper parallel configuration can significantly reduce test execution time, but requires careful consideration of thread safety and resource management.

Parallel Execution Modes

Suite-Level Parallelism

Run multiple <test> tags in parallel:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Parallel Suite" parallel="tests" thread-count="3">

<test name="Chrome Tests">
    <classes>
        <class name="com.example.tests.BrowserTest"/>
    </classes>
</test>

<test name="Firefox Tests">
    <classes>
        <class name="com.example.tests.BrowserTest"/>
    </classes>
</test>

<test name="Safari Tests">
    <classes>
        <class name="com.example.tests.BrowserTest"/>
    </classes>
</test>

</suite>

Class-Level Parallelism

Run test classes in parallel:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Parallel Classes" parallel="classes" thread-count="4">

<test name="All Tests">
    <classes>
        <class name="com.example.tests.UserServiceTest"/>
        <class name="com.example.tests.ProductServiceTest"/>
        <class name="com.example.tests.OrderServiceTest"/>
        <class name="com.example.tests.PaymentServiceTest"/>
    </classes>
</test>

</suite>

Method-Level Parallelism

Run test methods in parallel:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Parallel Methods" parallel="methods" thread-count="5">

<test name="Service Tests">
    <classes>
        <class name="com.example.tests.IndependentMethodsTest"/>
    </classes>
</test>

</suite>

Instance-Level Parallelism

Run test instances in parallel (useful with Factory):

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Parallel Instances" parallel="instances" thread-count="3">

<test name="Factory Tests">
    <classes>
        <class name="com.example.tests.FactoryGeneratedTest"/>
    </classes>
</test>

</suite>

Thread Pool Configuration

Basic Thread Configuration

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Thread Pool Suite" parallel="methods" thread-count="10"> <!-- Global thread pool configuration -->

<test name="Test Group 1" thread-count="5">
    <!-- Override for this specific test -->
    <classes>
        <class name="com.example.tests.Test1"/>
    </classes>
</test>

<test name="Test Group 2">
    <!-- Uses suite-level thread-count -->
    <classes>
        <class name="com.example.tests.Test2"/>
    </classes>
</test>

</suite>

Data Provider Parallel Execution

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="DataProvider Suite" data-provider-thread-count="20">

<test name="Data Driven Tests">
    <classes>
        <class name="com.example.tests.ParallelDataProviderTest"/>
    </classes>
</test>

</suite>

public class ParallelDataProviderTest {

@DataProvider(name = "largeDataSet", parallel = true)
public Object[][] provideLargeDataSet() {
    Object[][] data = new Object[100][2];
    for (int i = 0; i < 100; i++) {
        data[i] = new Object[]{"User" + i, "user" + i + "@example.com"};
    }
    return data;
}

@Test(dataProvider = "largeDataSet")
public void testWithParallelData(String name, String email) {
    System.out.println(Thread.currentThread().getName() + " - Testing: " + name);
    // Each data row runs in parallel
}

}

Thread Safety Patterns

Thread-Local Storage

import org.testng.annotations.*;

public class ThreadLocalTest {

// Thread-local storage for test-specific resources
private static ThreadLocal<WebDriver> driverThread = new ThreadLocal<>();
private static ThreadLocal<String> sessionThread = new ThreadLocal<>();

@BeforeMethod
public void setUp() {
    // Initialize thread-local resources
    driverThread.set(createWebDriver());
    sessionThread.set(generateSessionId());
}

@AfterMethod
public void tearDown() {
    // Clean up thread-local resources
    WebDriver driver = driverThread.get();
    if (driver != null) {
        driver.quit();
    }
    driverThread.remove();
    sessionThread.remove();
}

@Test
public void testParallelBrowser() {
    WebDriver driver = driverThread.get();
    String session = sessionThread.get();
    System.out.println("Thread: " + Thread.currentThread().getName() +
                      " Session: " + session);
    // Use thread-local driver
}

private WebDriver createWebDriver() {
    // Create browser instance
    return new ChromeDriver();
}

private String generateSessionId() {
    return UUID.randomUUID().toString();
}

}

Synchronized Shared Resources

import org.testng.annotations.*; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.ConcurrentHashMap;

public class SynchronizedResourceTest {

// Thread-safe counter
private static AtomicInteger testCounter = new AtomicInteger(0);

// Thread-safe collection
private static ConcurrentHashMap<String, String> sharedCache = new ConcurrentHashMap<>();

// Lock object for critical sections
private static final Object lock = new Object();

@Test(threadPoolSize = 5, invocationCount = 100)
public void testAtomicOperations() {
    int count = testCounter.incrementAndGet();
    System.out.println("Test count: " + count);
}

@Test(threadPoolSize = 3, invocationCount = 50)
public void testConcurrentMap() {
    String threadName = Thread.currentThread().getName();
    sharedCache.put(threadName, String.valueOf(System.currentTimeMillis()));
    // Thread-safe without explicit synchronization
}

@Test(threadPoolSize = 2, invocationCount = 10)
public void testSynchronizedBlock() {
    synchronized (lock) {
        // Critical section - only one thread at a time
        performCriticalOperation();
    }
}

private void performCriticalOperation() {
    // Operations that require exclusive access
}

}

Immutable Test Data

import java.util.Collections; import java.util.List; import java.util.Arrays;

public class ImmutableDataTest {

// Immutable test data - inherently thread-safe
private static final List<String> TEST_USERS = Collections.unmodifiableList(
    Arrays.asList("user1", "user2", "user3", "user4", "user5")
);

private static final Map<String, String> CONFIG = Collections.unmodifiableMap(
    Map.of(
        "url", "https://api.example.com",
        "timeout", "30000",
        "retries", "3"
    )
);

@Test(threadPoolSize = 5, invocationCount = 20)
public void testWithImmutableData() {
    // Safe to read from multiple threads
    int userIndex = ThreadLocalRandom.current().nextInt(TEST_USERS.size());
    String user = TEST_USERS.get(userIndex);
    String url = CONFIG.get("url");

    System.out.println(Thread.currentThread().getName() +
                      " - User: " + user + ", URL: " + url);
}

}

Test Isolation Patterns

Independent Test Methods

public class IndependentTestsExample {

// Each test method is completely independent
@Test
public void testFeatureA() {
    // Create its own resources
    UserService service = new UserService();
    User user = service.createUser("testA");
    assertNotNull(user);
    // Clean up
    service.deleteUser(user.getId());
}

@Test
public void testFeatureB() {
    // Completely separate from testFeatureA
    ProductService service = new ProductService();
    Product product = service.createProduct("testB");
    assertNotNull(product);
    service.deleteProduct(product.getId());
}

@Test
public void testFeatureC() {
    // No shared state with other tests
    OrderService service = new OrderService();
    Order order = service.createOrder();
    assertNotNull(order);
    service.cancelOrder(order.getId());
}

}

Isolated Database Tests

import org.testng.annotations.*;

public class IsolatedDatabaseTest {

private Connection connection;
private String testSchema;

@BeforeMethod
public void setUp() throws SQLException {
    // Create isolated schema for each test
    testSchema = "test_" + Thread.currentThread().getId() + "_" + System.currentTimeMillis();
    connection = DriverManager.getConnection(DB_URL, USER, PASSWORD);
    connection.createStatement().execute("CREATE SCHEMA " + testSchema);
    connection.setCatalog(testSchema);
    initializeTestData();
}

@AfterMethod
public void tearDown() throws SQLException {
    // Drop isolated schema
    connection.createStatement().execute("DROP SCHEMA " + testSchema + " CASCADE");
    connection.close();
}

@Test
public void testDatabaseOperation1() throws SQLException {
    // Operations in isolated schema
    PreparedStatement ps = connection.prepareStatement(
        "INSERT INTO users (name) VALUES (?)"
    );
    ps.setString(1, "TestUser1");
    ps.executeUpdate();

    ResultSet rs = connection.createStatement().executeQuery(
        "SELECT COUNT(*) FROM users"
    );
    rs.next();
    assertEquals(rs.getInt(1), 1);
}

@Test
public void testDatabaseOperation2() throws SQLException {
    // Completely isolated from testDatabaseOperation1
    PreparedStatement ps = connection.prepareStatement(
        "INSERT INTO products (name) VALUES (?)"
    );
    ps.setString(1, "TestProduct");
    ps.executeUpdate();
}

private void initializeTestData() throws SQLException {
    // Create tables in isolated schema
}

}

Parallel Execution with Dependencies

Preserving Order Within Groups

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Ordered Parallel" parallel="classes" thread-count="3">

<test name="Ordered Test" preserve-order="true">
    <classes>
        <!-- Classes run in parallel, methods in order -->
        <class name="com.example.tests.OrderedTest1">
            <methods>
                <include name="step1"/>
                <include name="step2"/>
                <include name="step3"/>
            </methods>
        </class>
        <class name="com.example.tests.OrderedTest2"/>
    </classes>
</test>

</suite>

Group Threading

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Group Threading" parallel="methods" thread-count="4" group-by-instances="true">

<test name="Instance Grouped">
    <classes>
        <class name="com.example.tests.InstanceGroupTest"/>
    </classes>
</test>

</suite>

public class InstanceGroupTest {

private String instanceId;

@Factory
public Object[] createInstances() {
    return new Object[] {
        new InstanceGroupTest("instance1"),
        new InstanceGroupTest("instance2"),
        new InstanceGroupTest("instance3")
    };
}

public InstanceGroupTest() {}

public InstanceGroupTest(String instanceId) {
    this.instanceId = instanceId;
}

@Test
public void step1() {
    System.out.println(instanceId + " - Step 1");
}

@Test(dependsOnMethods = "step1")
public void step2() {
    System.out.println(instanceId + " - Step 2");
}

@Test(dependsOnMethods = "step2")
public void step3() {
    System.out.println(instanceId + " - Step 3");
}

}

Performance Optimization

Optimal Thread Count

public class ThreadCountOptimization {

// Determine optimal thread count based on available resources
public static int getOptimalThreadCount() {
    int availableProcessors = Runtime.getRuntime().availableProcessors();

    // For CPU-bound tests
    int cpuBoundThreads = availableProcessors;

    // For I/O-bound tests (network, file, database)
    int ioBoundThreads = availableProcessors * 2;

    // For mixed workloads
    int mixedThreads = (int) (availableProcessors * 1.5);

    return mixedThreads;
}

}

Resource Pool Pattern

import java.util.concurrent.*;

public class ResourcePoolTest {

// Connection pool for parallel tests
private static BlockingQueue<Connection> connectionPool;

@BeforeSuite
public void setUpSuite() {
    int poolSize = 10;
    connectionPool = new ArrayBlockingQueue<>(poolSize);
    for (int i = 0; i < poolSize; i++) {
        connectionPool.offer(createConnection());
    }
}

@AfterSuite
public void tearDownSuite() {
    Connection conn;
    while ((conn = connectionPool.poll()) != null) {
        closeConnection(conn);
    }
}

@Test(threadPoolSize = 5, invocationCount = 50)
public void testWithPooledConnection() throws InterruptedException {
    Connection conn = connectionPool.take(); // Borrow
    try {
        // Use connection
        performDatabaseOperation(conn);
    } finally {
        connectionPool.offer(conn); // Return
    }
}

private Connection createConnection() {
    // Create database connection
    return null;
}

private void closeConnection(Connection conn) {
    // Close connection
}

private void performDatabaseOperation(Connection conn) {
    // Database operations
}

}

Reporting for Parallel Tests

Custom Reporter for Parallel Execution

import org.testng.*; import java.util.concurrent.ConcurrentHashMap;

public class ParallelTestReporter implements ITestListener {

private static ConcurrentHashMap<Long, List<String>> threadTestMap =
    new ConcurrentHashMap<>();

@Override
public void onTestStart(ITestResult result) {
    long threadId = Thread.currentThread().getId();
    threadTestMap.computeIfAbsent(threadId, k -> new CopyOnWriteArrayList<>())
                 .add(result.getName());
}

@Override
public void onFinish(ITestContext context) {
    System.out.println("\n=== Thread Distribution Report ===");
    threadTestMap.forEach((threadId, tests) -> {
        System.out.println("Thread " + threadId + ": " + tests.size() + " tests");
        tests.forEach(test -> System.out.println("  - " + test));
    });
    System.out.println("Total threads used: " + threadTestMap.size());
}

}

Timeout Configuration

public class TimeoutTest {

@Test(timeOut = 5000)
public void testWithTimeout() {
    // Fails if takes more than 5 seconds
}

@Test(timeOut = 10000, threadPoolSize = 3, invocationCount = 10)
public void testParallelWithTimeout() {
    // Each invocation has 10 second timeout
}

}

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="Timeout Suite" time-out="60000"> <!-- Suite-level timeout: 60 seconds total -->

<test name="Quick Tests" time-out="10000">
    <!-- Test-level timeout: 10 seconds for all tests in this group -->
    <classes>
        <class name="com.example.tests.QuickTest"/>
    </classes>
</test>

</suite>

Best Practices

• Design for independence - Tests should not depend on shared mutable state

• Use ThreadLocal for per-thread resources - Drivers, sessions, connections

• Prefer immutable data - Thread-safe by design

• Set appropriate thread counts - Based on resource availability

• Implement proper cleanup - Prevent resource leaks in parallel execution

• Use thread-safe collections - ConcurrentHashMap, CopyOnWriteArrayList

• Configure timeouts - Prevent hung tests from blocking threads

• Monitor thread distribution - Ensure balanced workload

• Test locally first - Verify thread safety before CI/CD

• Document thread safety requirements - Clear expectations for test authors

Common Pitfalls

• Shared mutable state - Causes race conditions and flaky tests

• Static fields without synchronization - Not thread-safe

• Resource contention - Too many threads competing for limited resources

• Order dependencies - Tests that assume execution order

• Missing cleanup - ThreadLocal resources not removed

• Insufficient isolation - Database tests affecting each other

• Too many threads - Overhead exceeds benefits

• Ignoring timeouts - Hung tests blocking execution

• Non-deterministic failures - Hard to reproduce parallel issues

• Improper connection pooling - Connection leaks or exhaustion

When to Use This Skill

• Reducing test suite execution time

• Configuring CI/CD parallel test execution

• Implementing thread-safe test infrastructure

• Designing parallel-friendly test architecture

• Troubleshooting parallel test failures

• Optimizing resource utilization in tests

• Building scalable test frameworks

• Implementing cross-browser parallel testing