In the world of test automation, Selenium stands out as one of the most widely used tools for web application testing. It provides powerful capabilities for automating browser actions, ensuring that applications function as expected across different environments. However, running automated tests alone isn’t enough—understanding test results efficiently is just as important. This is where test reporting comes into play.
Why is test reporting crucial in Selenium automation?
When executing multiple test cases, especially in large-scale projects, it’s essential to have clear and structured reports that provide insights into:
- Test execution status – Which tests passed, failed, or were skipped?
- Error details – What caused a test failure? Logs, screenshots, and stack traces help in debugging.
- Execution trends – Analyzing patterns in test failures over time.
- Collaboration & accountability – Sharing test results with teams, stakeholders, and CI/CD pipelines for better decision-making.
Without effective reporting, identifying and fixing issues can become time-consuming and inefficient. That’s why integrating a reliable reporting tool with Selenium automation is essential for better test visibility, faster debugging, and improved software quality.
In this blog, we’ll explore the top 10 reporting tools for Selenium, their key features, and how they can enhance your test automation process.
- Importance of Reporting in Selenium Automation
- Popular Reporting Tools for Selenium
- TestNG
- ReportNG
- Surefire Report
- Junit Report
- Serenity Report
- Serenity BDD Setup for Reporting
- Cucumber Report
- Why Are Cucumber Reports Important?
- Step-by-Step: Setting Up Cucumber Reports in Java
- How to Choose the Right Reporting Tool for Selenium
- Conclusion:
Importance of Reporting in Selenium Automation
Why Reporting Matters in Selenium Automation?
Selenium automation is widely used for validating web applications, but without clear and structured test reports, identifying issues can be time-consuming. Reports bridge the gap between test execution and analysis, ensuring that teams can track test outcomes, debug failures efficiently, and improve overall test effectiveness.
Imagine executing hundreds of automated test cases—how do you quickly determine which tests passed, which failed, and why? Manually reviewing logs is impractical. This is where test reporting tools come in, providing structured and visualized test execution data.
The Role of Reports in Test Execution Analysis
A good reporting tool provides the following crucial insights:
- Test Case Execution Summary – A structured view of passed, failed, and skipped tests, making it easy to analyze the results.
- Root Cause Identification – Reports display logs, stack traces, and failure reasons to help pinpoint defects quickly.
- Test Execution Time & Performance Analysis – Helps in identifying slow test cases or performance bottlenecks.
- Regression Testing Insights – Compares test execution trends across different builds to detect regressions.
- Test Run History & Trends – Allows tracking of execution patterns over multiple test cycles.
Key Benefits of Using Reporting Tools
Now, let’s explore the key advantages of using reporting tools in Selenium automation:
Quick Identification of Test Failures
Without reports:
- Finding failed tests requires manually checking console logs, which can be overwhelming.
- Teams spend more time searching for errors rather than fixing them.
With reports:
- Reports provide a clear summary of failed test cases along with detailed failure reasons.
- Failure traces, logs, and screenshots help in quickly diagnosing problems.
Example:
Imagine running an e-commerce checkout test. If the payment page is broken, a report will immediately flag the failure with an error message like.
ElementNotFoundException: Unable to locate element with ID ‘payment-button’
Easy Debugging with Screenshots and Logs
Without reports:
- Debugging requires going through long logs or re-running tests to reproduce errors.
With reports:
- Advanced reporting tools capture screenshots on failure.
- They also store execution logs and stack traces, helping testers quickly diagnose and fix issues.
Example:
A Selenium test for a login page fails because the password field was not found. A report with a screenshot will show that the element was hidden due to a UI bug.
Popular tools like Allure and Extent Reports provide visual logs, screenshots, and step-by-step execution details.
Enhanced Collaboration with Stakeholders
Test automation is not just for QA engineers—other stakeholders like project managers, business analysts, and developers also need insights into test execution.
Without reports:
- Teams rely on raw test logs, which are difficult to interpret.
- Non-technical stakeholders have no visibility into test execution.
With reports:
- Reports are generated in user-friendly formats (HTML, PDF, JSON).
- They can be automatically emailed or integrated with CI/CD tools like Jenkins, Azure DevOps, and GitHub Actions.
Example:
- A manager reviewing the weekly automation test results can check an HTML
- report showing:
- 100 tests executed
- 90 tests passed
- 10 tests failed with details
This improves decision-making and communication between teams.
Better Visibility into Automation Coverage
For long-term test automation success, it’s essential to track test coverage and execution trends.
Without reports:
- Test coverage analysis is difficult.
- No insights into test case stability and effectiveness.
With reports:
- Visual charts and graphs show execution trends over time.
- Helps in identifying frequent test failures and unstable test cases.
Example:
- A project team tracks Selenium test execution over 10 sprints.
- A report shows:
- Pass rate increased from 85% to 95%
- Flaky test failures reduced by 50%
- This helps teams optimize test cases and improve automation efficiency.
Popular Reporting Tools for Selenium
TestNG
A TestNG report is a structured and detailed summary of test execution results, automatically generated when running TestNG-based test cases in a Selenium or Java testing framework. These reports help QA engineers and testers analyze test results, failures, execution times, skipped tests, and errors in an easy-to-read format.
TestNG generates two types of reports by default:
- Console Report – Displays test execution results in the terminal.
- HTML Report – A structured report available in the test-output folder.
Apart from the default reports, TestNG also supports custom reporting using:
- TestNG Listeners
- Reporter Class
- Third-Party Plugins (Extent Reports, Allure Reports, etc.)
Types of TestNG Reports
Default TestNG Reports
TestNG generates the following reports automatically after test execution:
Console Report
- Displays test execution status directly in the terminal/IDE.
- Includes PASSED, FAILED, SKIPPED tests.
- Shows execution time and errors (if any).
HTML Reports (index.html & emailable-report.html
- Generated inside the test-output directory.
- index.html – Provides a detailed breakdown of test execution.
- emailable-report.html – A summary report that can be shared via email.
Location of Reports:
After running the tests, find them at:
- test-output/index.html
- test-output/emailable-report.html
Example Output:
A sample index.html report contains:
- Suite Name
- Test Execution Time
- Number of Passed, Failed, and Skipped Tests
- Error Stack Trace for Failures
Custom Reports in TestNG
Apart from default reports, TestNG allows generating custom reports using:
TestNG Listeners (ITestListener)
- Listens to test execution events and allows us to log results.
- Helps in generating logs and tracking test execution in real-time.
Example: Custom Listener for Reporting
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class CustomTestNGListener implements ITestListener {
@Override
public void onStart(ITestContext context) {
System.out.println("Test Suite Started: " + context.getName());
}
@Override
public void onTestSuccess(ITestResult result) {
System.out.println("Test Passed: " + result.getName());
}
@Override
public void onTestFailure(ITestResult result) {
System.out.println("Test Failed: " + result.getName());
}
@Override
public void onTestSkipped(ITestResult result) {
System.out.println("Test Skipped: " + result.getName());
}
}
Usage in testng.xml
<listeners>
<listener class-name="listeners.CustomTestNGListener"/>
</listeners>
Using TestNG Reporter Class (Reporter.log)
- Allows logging custom messages in the TestNG report.
Example: Logging Custom Messages
import org.testng.Reporter;
import org.testng.annotations.Test;
public class TestNGReportExample {
@Test
public void testMethod() {
Reporter.log("Test Started: Verifying Login Functionality", true);
System.out.println("Executing Test...");
Reporter.log("Test Completed Successfully", true);
}
}
Output in emailable-report.html
- Test Started: Verifying Login Functionality
- Executing Test…
- Test Completed Successfully
Extent Reports
Extent Reports is a feature-rich, interactive test reporting library designed for Selenium, TestNG, JUnit, and Cucumber frameworks. It helps in visualizing test execution results in an attractive and structured format.
Key Features of Extent Reports:
- Graphical Representation of Test Results:
- Provides pie charts, bar graphs, and trend analysis for test execution statistics.
- Helps visualize pass/fail trends over multiple test runs.
- Provides pie charts, bar graphs, and trend analysis for test execution statistics.
- Step-by-Step Execution Logs:
- Captures test steps, logs, and timestamps to track execution flow.
- Logs messages at different levels: INFO, PASS, FAIL, WARNING, SKIP, ERROR, DEBUG.
- Captures test steps, logs, and timestamps to track execution flow.
- Screenshots and Attachments:
- Allows attaching screenshots of test failures.
- Supports adding logs, videos, and other artifacts for debugging.
- Allows attaching screenshots of test failures.
- Customizable Themes and Categories:
- Supports dark/light modes and different layouts.
- Categorization of test cases based on modules, priorities, or functionalities.
- Supports dark/light modes and different layouts.
- Parallel Execution Support:
- Works well with TestNG parallel execution, displaying concurrent test results efficiently.
- Works well with TestNG parallel execution, displaying concurrent test results efficiently.
- Integration with CI/CD Tools:
- Compatible with Jenkins, GitLab, GitHub Actions, and Azure DevOps.
- Generates reports in HTML, PDF, or JSON formats for sharing and analysis.
- Compatible with Jenkins, GitLab, GitHub Actions, and Azure DevOps.
Extent Report Example in TestNG with Selenium:
Add Extent Reports dependency in pom.xml
<!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports</artifactId>
<version>5.1.2</version>
</dependency>
ExtentReportExample.java
import com.aventstack.extentreports.*;
import com.aventstack.extentreports.reporter.ExtentSparkReporter;
import org.testng.annotations.*;
public class ExtentReportExample {
ExtentReports extent;
ExtentTest test;
@BeforeTest
public void setupReport() {
ExtentSparkReporter spark = new ExtentSparkReporter("ExtentReport.html");
extent = new ExtentReports();
extent.attachReporter(spark);
}
@Test
public void testPass() {
test = extent.createTest("Test Case 1").pass("Test Passed");
}
@AfterTest
public void tearDown() {
extent.flush(); // Generate the report
}
}
Best For:
- UI automation reporting with detailed logs.
- TestNG, JUnit, and Cucumber-based frameworks.
- Customizing and enhancing the visual appeal of test reports.
To learn more about these reports, check out our previous blog: Effortless Test Reporting in Test Automation With Selenium.
ReportNG
Test reporting plays a crucial role in any test automation process, ensuring that test results are documented effectively. Reports should be generated automatically once the test execution is complete.
ReportNG is a useful tool for generating test reports in both HTML and XML formats for Selenium WebDriver (Java) projects.
In this guide, we will walk through the process of integrating ReportNG with an existing Selenium WebDriver + TestNG test suite in IntelliJ IDEA.
Before proceeding, it is recommended that you have prior knowledge of integrating Selenium WebDriver with TestNG.
Additionally, you can customize the test reports by modifying the stylesheets to match your preferences.
What is ReportNG?
ReportNG is an advanced HTML reporting plugin for TestNG that enhances test execution reports with better visualization. It replaces the default TestNG reports with a cleaner, color-coded, and more readable format.
Why Use ReportNG?
- The default TestNG reports are plain and difficult to read.
- ReportNG provides a structured and color-coded HTML report.
- It supports JUnit XML output, making it compatible with Jenkins and CI/CD tools.
- You can customize the report’s look using a custom CSS file.
- Integrates easily with Selenium WebDriver and Maven.
Prerequisites
Before setting up ReportNG, ensure that you have:
- Java Development Kit (JDK) installed.
- Maven set up for dependency management.
- TestNG framework installed.
- Selenium WebDriver configured.
Add ReportNG Dependencies in pom.xml
To use ReportNG with TestNG, you need to add the ReportNG and TestNG dependencies in your Maven project.
Include the following dependencies in pom.xml:
<dependencies>
<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.29.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.11.0</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.uncommons/reportng -->
<dependency>
<groupId>org.uncommons</groupId>
<artifactId>reportng</artifactId>
<version>1.1.4</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.inject/guice -->
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>7.0.0</version>
</dependency>
Adding maven-surefire-plugin to the pom.xml
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.2</version> <!-- Latest stable version as of October 2024 -->
<configuration>
<properties>
<!-- Disable default TestNG listeners to prevent duplicate reports -->
<property>
<name>usedefaultlisteners</name>
<value>false</value>
</property>
</properties>
</configuration>
</plugin>
</plugins>
</build>
Adding listeners
Add following listeners to the TestNG.xml.
<listeners>
<listener class-name="org.uncommons.reportng.HTMLReporter"/>
<listener class-name="org.uncommons.reportng.JUnitXMLReporter"/>
</listeners>
What Do These Listeners Do?
- HTMLReporter generates color-coded, structured HTML reports.
- JUnitXMLReporter creates JUnit-compatible XML reports (useful for Jenkins integration).
Run the test suite
- Click run→ Run <Testsuite-name> menu item in IntelliJ IDEA
- Also you should be able to run the testing.xml from the context menu too.
Checking the test reports
Generated reports will be in the target folder. You can open the report file from IntelliJ IDEA.
- Navigate to the Test-output –> html folder
- Select index.html file
- Right click on the index.html file
- Select Open in Browser –> <Web browser>
Surefire Report
What is a Surefire Report?
The Maven Surefire Report is an automated test execution report generated using the Maven Surefire Plugin. It is widely used in Selenium-based automation projects with TestNG or JUnit frameworks.
This report provides a detailed analysis of test execution, including pass/fail status, execution time, and error messages for failed tests. The reports are generated in XML, plain text, and HTML formats, making them useful for both debugging and sharing results with the team.
Why is the Surefire Report Important?
The Surefire report plays a crucial role in test automation by offering:
- Automated Test Reporting – Eliminates manual effort in tracking test execution.
- Easy Debugging – Highlights test failures, errors, and stack traces.
- CI/CD Integration – Works seamlessly with Jenkins, GitHub Actions, GitLab CI/CD, and other CI/CD pipelines.
- Support for Multiple Frameworks – Compatible with TestNG and JUnit.
- Readable Report Formats – Available in XML, plain text, and HTML, making it easier to analyze.
How Does Maven Surefire Work?
The Maven Surefire Plugin executes test cases in the project and generates structured reports based on the test execution results. The process includes:
- Running Automated Tests – Surefire executes test cases written in TestNG or JUnit.
- Capturing Test Results – The plugin records pass/fail status, errors, and execution time.
- Generating Reports – The reports are stored in the target/surefire-reports/ directory.
After test execution, the reports contain:
- Number of tests executed
- Total execution time
- Pass/fail/skipped test cases
- Error messages and stack traces for failed tests
Types of Surefire Reports
The Maven Surefire Plugin generates reports in three formats:
XML Reports (TEST-*.xml)
Location: target/surefire-reports/TEST-*.xml
Contains structured test execution details
Useful for integrating test results into CI/CD pipelines
Helps in debugging test failures
Plain Text Reports (*.txt)
Location: target/surefire-reports/*.txt
Provides test results in simple text format
Contains test case names, status, and execution time
Helpful for quick review of test execution
HTML Report (emailable-report.html)
Location: target/surefire-reports/emailable-report.html
An interactive summary report
Easy to read and share via email
Ideal for stakeholders and QA teams
How to Generate a Surefire Report?
Add Maven Surefire Plugin to pom.xml
To generate Surefire reports, you need to add the Maven Surefire Plugin in your pom.xml file:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.1.2</version> <!-- Use the latest version -->
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
This plugin ensures that test cases are executed and reports are generated under target/surefire-reports/.
Add Dependencies for TestNG or JUnit
Since Surefire supports both TestNG and JUnit, add the required dependency in pom.xml:
For TestNG Users
<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.10.2</version>
<scope>test</scope>
</dependency>
For JUnit Users
<!-- https://mvnrepository.com/artifact/junit/junit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
Create Test Cases in Selenium + TestNG
Create a sample test case under src/test/java
Run Tests Using Maven
To execute tests and generate Surefire reports, run the following command:
- mvn clean test
Tests will be executed, and reports will be generated in target/surefire-reports/.
Verify Generated Reports
After execution, check the target/surefire-reports/ folder. You will find:
- TEST-SampleTest.xml → XML report
- emailable-report.html → HTML report
- surefire-summary.txt → Plain text report
To view the HTML report, open target/surefire-reports/emailable-report.html in a browser.
HTML [Emailable report]
Junit Report
JUnit reports are XML-based reports generated after executing test cases using JUnit. These reports provide insights into test execution results, including passed, failed, and skipped tests. JUnit reports are widely used in continuous integration (CI) tools like Jenkins, GitHub Actions, and GitLab CI/CD for tracking test execution trends.
- Test Suite Name
- Total Tests Run
- Passed, Failed, and Skipped Test Counts
- Execution Time for Each Test
- Detailed Failure Messages (if any)
Generating JUnit Reports Using Maven Surefire Plugin
Add Dependencies in pom.xml
To generate JUnit reports, ensure that the JUnit and Maven Surefire Plugin are included in your pom.xml:
<!-- https://mvnrepository.com/artifact/junit/junit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
Configure Maven Surefire Plugin
The Maven Surefire Plugin is responsible for executing JUnit tests and generating reports.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.1.2</version> <!-- Use the latest version -->
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
Running JUnit Tests and Generating Reports
Run the following Maven command to execute the tests and generate JUnit reports:
- mvn clean test
This will execute the test cases
The reports will be saved in:
target/surefire-reports/TEST-*.xml
Structure of a JUnit Report
JUnit reports follow an XML format, containing details about the test execution.
<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated by org.testng.reporters.JUnitReportReporter -->
<testsuite hostname="JTPC03" failures="0" tests="5" name="login.LoginTest" time="1.631" errors="1" timestamp="2025-04-02T18:53:20 IST" skipped="4">
<testcase classname="login.LoginTest" name="setUp" time="1.631">
<error message="element not interactable
(Session info: chrome=134.0.6998.178)
Build info: version: '4.27.0', revision: 'd6e718d134'
Key Elements in JUnit XML Reports:
Element | Description |
<testsuite> | Represents a collection of test cases |
name | Test suite name |
tests | Total number of tests executed |
failures | Number of failed test cases |
errors | Number of tests that encountered errors |
skipped | Number of skipped tests |
<testcase> | Represents an individual test case |
<failure> | Contains failure message and stack trace if a test fails |
<skipped> | Marks a test as skipped |
Benefits of JUnit Reports
- Test Execution Summary – Provides a clear overview of test results
- Easier Debugging – Shows failure messages and stack traces for quick analysis
- Integration with CI/CD – Supports automation tools like Jenkins, GitHub Actions, and GitLab
- Customizable Reporting – Can be converted into HTML reports for better readability.
Serenity Report
What is Serenity BDD?
Serenity BDD is an open-source library for writing automated acceptance and regression tests. It supports various test runners such as:
- JUnit
- Cucumber
- RestAssured (for API testing)
The main features of Serenity BDD:
- Helps write tests in a more maintainable, readable way.
- Integrates with Selenium WebDriver for UI automation.
- Built-in support for rich, interactive HTML reports.
These reports are one of Serenity’s standout features let’s explore them in depth.
Why Serenity Reports Matter in Test Automation
Here’s why Serenity BDD reports are so powerful:
Feature | Benefit |
Step-by-step reports | See exactly what happened during test execution |
Screenshots | Capture every UI interaction (optional) |
Requirement mapping | Link tests to user stories or epics |
Tag grouping | Filter tests by feature, tag, or requirement |
Error visibility | View stack traces, error messages, and logs |
Coverage metrics | See test coverage vs. defined requirements |
These reports help testers, business analysts, and developers (or non-technical team members) understand the status of the application at a glance.
Serenity BDD Setup for Reporting
Let’s walk through a basic setup using Maven + JUnit.
Tools Required
- Java (JDK 11+ recommended)
- Maven
- IntelliJ or Eclipse IDE
- Chrome/Firefox driver (for Selenium)
Maven Project Structure
pom.xml Configuration
<modelVersion>4.0.0</modelVersion>
<groupId>your.group.id</groupId>
<artifactId>your-artifact-id</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>18</maven.compiler.source>
<maven.compiler.target>18</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<serenity.version>3.6.12</serenity.version>
</properties>
<dependencies>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>6.11.0</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>6.11.0</version>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-core</artifactId>
<version>${serenity.version}</version>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-cucumber6</artifactId>
<version>2.6.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>18</source>
<target>18</target>
</configuration>
</plugin>
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>4.0.15</version>
<executions>
<execution>
<id>serenity-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
serenity.conf
serenity {
webdriver {
driver = "chrome"
}
project.name = "My Serenity BDD Project"
outputDirectory = "target/site/serenity-reports"
serenity.take.screenshots = FOR_FAILURES
}
How to Generate Serenity Reports
Step 1: Run Your Tests with Maven
Hit this command into your project terminal “mvn clean verify”
- mvn: This is the Maven command.
- clean: Deletes the previous build outputs (like compiled classes and reports) to ensure a fresh build.
- verify: This phase runs your test cases and ensures everything meets the quality criteria. If you’re using Serenity, it will also trigger Serenity’s built-in reporting mechanism.
So when you run mvn clean verify, it compiles the code, runs the tests, and prepares the Serenity reports.
Locate the Serenity Report
“target/site/serenity/index.html”
- After the tests are executed, Serenity automatically generates a visual HTML report.
- The report is located at: target/site/serenity/index.html
- target/ is the default build directory for Maven.
- site/serenity/ is where Serenity places its output.
- target/ is the default build directory for Maven.
- index.html is the main file you open in a browser to view the test results.
Cucumber Report
Why Are Cucumber Reports Important?
Cucumber reports offer:
- A visual summary of test execution
- A record of which scenarios passed or failed
- Error stack traces for debugging
- Execution duration details
- Screenshot support for failed tests (with customization)
Whether you’re running tests locally or in a CI/CD pipeline, having good reports helps both QA and business teams track progress effectively.
Step-by-Step: Setting Up Cucumber Reports in Java
Set Up a Basic Cucumber Framework
src/
├── test/
│ ├── java/
│ │ ├── stepDefinitions/
│ │ └── runners/
│ ├── resources/
│ │ └── features/
Pom.xml
Add Required Dependencies to pom.xml
<dependencies>
<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-java -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>7.18.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-junit -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>7.18.0</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.24.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/junit/junit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
</dependencies>
Generate Advanced Report (Optional but Recommended)
Add Masterthought Cucumber Reporting Plugin in pom.xml
Inside <build><plugins> section:
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>5.7.4</version>
<executions>
<execution>
<id>cucumber-report</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>My BDD Automation Report</projectName>
<outputDirectory>${project.build.directory}/advanced-report</outputDirectory>
<cucumberOutput>${project.build.directory}/cucumber-reports/CucumberTestReport.json</cucumberOutput>
</configuration>
</execution>
</executions>
</plugin>
Create Feature File
Feature: Login feature
Scenario: Successful login
Given user is on login page
When user enters valid credentials
Then user should be navigated to the home page
Create Step Definitions
package stepDefinition;
import io.cucumber.java.en.*;
public class LoginSteps {
@Given("user is on login page")
public void user_on_login_page() {
System.out.println("User is on the login page");
}
@When("user enters valid credentials")
public void user_enters_valid_credentials() {
System.out.println("User enters username and password");
}
@Then("user should be navigated to the home page")
public void user_navigated_to_home_page() {
System.out.println("User successfully logged in");
}
}
Cucumber Test Runner
package runner;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
@RunWith(Cucumber.class)
@CucumberOptions(
features = "src/test/resources/featureFiles/registration.feature",
glue = "stepDefinition",
plugin = {"pretty",
"html:target/cucumber-reports/cucumber-html-report.html",
"json:target/cucumber-reports/CucumberTestReport.json"
},
monochrome = true
)
public class TestRunner {
}
Run the Tests
In your terminal, navigate to your project folder and run:
- mvn clean test
View the Reports
After the test run completes, reports will be generated in:
- target/cucumber-reports/cucumber-html-report.html
- target/cucumber-reports/cucumber-report.json
- target/cucumber-reports/cucumber-report.xml
Open the HTML file in a browser to see the readable Cucumber report.
Passed Test cases
How to Choose the Right Reporting Tool for Selenium
Choosing the right reporting tool is essential to gain insights from your Selenium test execution and improve overall testing efficiency. Here are the key factors to consider:
1. Compatibility with Your Test Framework
- Make sure the tool supports your test runner — TestNG, JUnit, or Cucumber.
- Tools like ExtentReports and Allure offer built-in adapters for most major frameworks.
2. Graphical Representation and Interactivity
- A good reporting tool should provide easy-to-read charts, tables, and interactive dashboards.
- Tools like Allure Reports and ExtentReports stand out for their clean, interactive UIs that clearly show test statuses, durations, and history.
3. Support for Screenshots and Logs
- Choose a tool that can attach screenshots on failure, browser logs, and step-by-step traces.
- This is critical for debugging issues quickly, especially in UI automation.
4. CI/CD Integration
- Ensure the reporting tool integrates easily with CI/CD tools like Jenkins, GitLab CI, or Azure DevOps.
- It should support command-line generation and publishing options to automate reporting in pipelines.
5. Customization Flexibility
- Look for tools that allow you to add custom labels, environment info, metadata, or tags.
- Customizing the report output makes it easier for stakeholders to interpret test results in context.
Popular Reporting Tools You Can Consider:
Tool | Best For |
ExtentReports | Interactive and detailed HTML reports with screenshots |
Allure Reports | Beautiful visual reports with test history and attachments |
Serenity BDD | Ideal for behavior-driven tests and living documentation |
ReportNG | Lightweight and simple, works with TestNG |
Conclusion:
Choosing the right reporting tool for Selenium is more than just adding visuals to your test results—it’s about making your test outcomes clear, actionable, and accessible to all stakeholders. Whether you’re using TestNG, JUnit, or Cucumber, an effective reporting tool helps bridge the gap between testers and business teams by providing detailed insights, visual dashboards, failure analysis, and seamless CI/CD integration.
In the end, the best reporting tool is the one that fits your testing framework, project needs, and team workflow. Whether you go with ExtentReports for interactive HTML reports, Allure for rich visualization and history, or Serenity for BDD-style documentation, ensure it aligns with your test goals and delivery pipeline.
Investing time in setting up the right reporting tool pays off in better visibility, faster debugging, and higher confidence in your automation efforts.
Witness how our meticulous approach and cutting-edge solutions elevated quality and performance to new heights. Begin your journey into the world of software testing excellence. To know more refer to Tools & Technologies & QA Services.
If you would like to learn more about the awesome services we provide, be sure to reach out.
Happy Testing 🙂