Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Run Results docs as per updated UI #552

Merged
merged 1 commit into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions src/left-nav-title.json
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@
"/docs/reports/runs/rerun/": "ReRun"
},
"drill-down-reports": {
"/docs/reports/runs/drill-down-reports/": "Drill-down Reports"
"/docs/reports/runs/drill-down-reports/": "Test Plan Run Results"
},
"test-machine-reports": {
"/docs/reports/runs/test-machine-reports/": "Test Machine Level Reports"
Expand All @@ -370,7 +370,7 @@
"/docs/reports/runs/test-case-reports/": "Test Case Reports"
},
"filter-custom-reports": {
"/docs/reports/runs/filter-custom-reports": "Filters & Custom Reports"
"/docs/reports/runs/filter-custom-reports": "Filters on Run Results"
},
"export-report-junit": {
"/docs/reports/runs/export-report-junit/": "Export Reports - JUnit"
Expand Down Expand Up @@ -690,5 +690,6 @@
"generate-tdp": {"/docs/genai-capabilities/generate-tdp/": "Generate TDP Using Copilot"},
"smtp-configuration": {"/docs/configuration/smtp-configuration/": "SMTP Configuration"},
"generate-wda-file": {"/docs/configuration/generate-wda-file/": "Create WDA File" },
"environments": {"/docs/api/environments/": "Manage Environments"}
"environments": {"/docs/api/environments/": "Manage Environments"},
"runs": {"/docs/reports/runs/": "Results & Reports"}
}
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-report-pdf.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ In Testsigma, you can effortlessly export PDF reports with screenshots for your

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.

[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-report-xls.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ Effortlessly generate detailed Excel reports for your test runs in Testsigma. Th

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.

[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-reports-junit.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ You can conveniently export your test plan reports in XML format, specifically i

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.

[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
22 changes: 11 additions & 11 deletions src/pages/docs/reports/runs/custom-pdf-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,17 @@ Custom PDF Report Generator in Testsigma allows you to create detailed and custo

---

## **Prerequisites**

Before you begin, ensure you have the following:

- An [API Key ](https://testsigma.com/docs/reports/runs/filter-custom-reports/) from your Testsigma account.

- **Java 17** or a newer version installed on your system.

- You can request the **JAR file** from the Customer Support team and download it before executing a Custom PDF Report Generation.

- An **HTML template** for the PDF report. [Click here](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Custom_Report_Template.txt) to download the HTML template.
> ## **Prerequisites**
>
> Before you begin, ensure you have the following:
>
> - An [API Key ](https://testsigma.com/docs/reports/runs/filter-custom-reports/) from your Testsigma account.
>
> - **Java 17** or a newer version installed on your system.
>
> - You can request the **JAR file** from the Customer Support team and download it before executing a Custom PDF Report Generation.
>
> - An **HTML template** for the PDF report. [Click here](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Custom_Report_Template.txt) to download the HTML template.

---

Expand Down
170 changes: 108 additions & 62 deletions src/pages/docs/reports/runs/drill-down-reports.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Drill Down Reports in Testsigma"
metadesc: "Drill-down reports let you access data at all layers. Learn how to use drill-down reports in the test execution results of automated tests in Testsigma"
title: "Test Plan Run Results in Testsigma"
metadesc: "View and download reports from the Run Results page in Testsigma | This page provides detailed results at the test cases, suites, and machines levels."
noindex: false
order: 13.23
page_id: "Drill Down Reports in Testsigma"
page_id: "Test Plan Run Results"
warning: false
contextual_links:
- type: section
Expand All @@ -12,117 +12,163 @@ contextual_links:
name: "Prerequisites"
url: "#prerequisites"
- type: link
name: "How to Use Drill-down Reports in Testsigma"
url: "#how-to-use-drill-down-reports-in-testsigma"
name: "Steps to View Test Suite Results"
url: "#steps-to-view-test-suite-results"
- type: link
name: "Steps to Generate Test Case Reports"
url: "#steps-to-generate-test-case-reports"
name: "Steps to View Test Case Results"
url: "#steps-to-view-test-case-results"
- type: link
name: "Steps to Generate Test Suite Reports"
url: "#steps-to-generate-test-suite-reports"
name: "Steps to View Test Machine Results"
url: "#steps-to-view-test-machine-results"
- type: link
name: "Steps to Generate Test Machine Reports"
url: "#steps-to-generate-test-machine-reports"
name: "Test Runs in Run Results"
url: "#test-runs-in-run-results"
- type: link
name: "Additional Actions on Results"
url: "#additional-actions-on-results"
name: "Overview of Tests in Run Results"
url: "#overview-of-tests-in-run-results"
- type: link
name: "Exporting the Test Reports"
url: "#exporting-the-test-reports"
- type: link
name: "Available Shortcuts in Run Results"
url: "#available-shortcuts-in-run-results"
---

---

The users will have better satisfaction when the application does not have bugs, technical glitches, performance issues, or usability issues. When these issues increase, the quality decreases, which results in bad UX. That's when drill-down reports come in handy.
In Testsigma, you can view and download reports from the Run Results page. This page provides results at the test cases, suites, and machines levels and also helps you check test runs, view the run overview, and see the count of passed/failed tests along with the reasons for failure. This article provides an overview of the Run Results page in Testsigma.

In Testsigma, you can view/download reports at multiple levels and find specific details, like the number of tests failed and the reason for failures or machines causing more problems. You can assess the application's quality and address issues from this data.

---

## **Prerequisites**
> ## **Prerequisites**
>
> Before you begin, make sure you have created and executed a [test plan](https://testsigma.com/docs/test-management/test-plans/overview/).

- You should know how to [create a test case](https://testsigma.com/docs/test-cases/manage/add-edit-delete/#creating-a-test-case).
---

- You should know how to [create a test suite](https://testsigma.com/docs/test-management/test-plans/overview/#steps-to-create-a-test-plan).
## **Steps to View Test Suite Results**

- You should know how to [create a test plan](https://testsigma.com/docs/test-management/test-suites/overview/#creating-a-test-suit).
1. From the left navigation bar, go to **Run Results** and click on the test plan for which you want to check the results.
![Test Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Test_Suites.png)

---
2. By default, you’ll see the results at test suite level.
![Results at Test Suite Level](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Default_Page.png)

## **How to Use Drill-down Reports in Testsigma:**
3. You can expand the test suite to check test case results inside the suite.
![Expand Test Suite](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Expan_Test_Suite_Run_Results.png)

4. The symbols shown with the arrow in the image below indicate the number of test suites running in parallel and the number of test cases running in parallel within each test suite.
![Parallel Settings Indicators](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Parallel_Settings_In_Test_Suites.png)

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
---


2. From **Test Plan Result** page, you can view **Test Case** reports.
![Test Case Report](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtcr.png)
## **Steps to View Test Case Results**

1. On the **Run Results** page, select **Test Cases** from the dropdown menu.
![Test Cases](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Test_Cases.png)

[[info | **NOTE**:]]
| You can also view reports at test suite and test machine level by clicking on **Test Suite** and **Test Machine**.
| ![Reoprts](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtstm.png)
2. The page displays all test cases from all test suites.
![Test Cases in all Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Cases_In_All_Suites.png)

3. You can choose the different runs from the dropdown to see how the results have changed for different builds.
![Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrrabuilds.png)
3. You can hover over a failed test case to view a brief description of the reason for failure.
![Failure Dialog Box](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Failure_Dialog_Test_Cases.png)

4. Click a test case to view its detailed results.

---
5. On the **Test Case Results** page, you can view all the test steps along with the **Data, Settings, Metadata**, and **Step Details**.
![Detailed Result of a Test Case](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Detailed_Test_Case_Result.png)

6. Click **More details** to access additional information.
![More Details](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/More_Details_On_Test_Case_Result.png)

## **Steps to Generate Test Case Reports**
---

## **Steps to View Test Machine Results**

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
1. On the **Run Results** page, select **Test Machines** from the dropdown menu.
![Test Machine Run Results](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Machine_Run_Results.png)

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Case** to view the results grouped by **Test Cases**.
![Test Case Report](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtcr.png)
2. The page displays all test machines included in the test plan.
![All Test Machines](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/All_Test_Machines_Run_Results.png)

3. Click a test machine to expand and view the test suites it contains.
![Expand Machine](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Expand_Test_Machine_Run_Result.png)

---

## **Test Runs in Run Results**

1. From the **Test Runs** panel, select different runs to compare how the results have changed.
![Test Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Runs_In_Run_Results.png)

## **Steps to Generate Test Suite Reports**
2. Click **Rerun** in the top-right corner to rerun the test plan by selecting the appropriate rerun options.
![Rerun](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ReRun_from_Run_Results.png)

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
[[info | **NOTE**:]]
| 1. Only 10 reruns are available for each build.
| ![Rerun Limit](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Rerun_Limits_Run_Results.png)
|
| 2. Once you click **Rerun**, you'll see the following options.
| - **All Test Cases:** This will rerun all the test cases in the selected run.
| - **All Failed Test Cases:** This will rerun all the failed test cases in the selected run.
| - **Select Cases for Re-Run:** This will allow you to select the test cases you want to rerun.
|
| 3. Click **Start execution** to rerun the test plan.

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Suite** to view the results grouped by **Test Suites**.
![Test Suite](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtestsuite.png)

3. You can also check test case results inside test suite by clicking on the **Test suite** of your choice.
![Test Case in Test Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtstclr.png)
1. To compare the current rerun result with the previous result, click **Compare Run**.
![Compare Test Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Compare_Runs_In_Run_Results.png)

2. This opens the **Rerun** overlay, where you can view a detailed comparison at all levels.
![Rerun Overlay](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Compare_Runs_Overlay.png)

---

## **Steps to Generate Test Machine Reports**
## **Overview of Tests in Run Results**

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
1. The **Run Overview** section of the **Run Results** page is dynamic and provides a summary of tests at all levels for the selected run.
![Run Overview](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Run_Results.png)

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Machine** to view the results grouped by **Test Machines**.
![Test Machine Reports](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtmr.png)
2. This section displays the **Last Run, Test Plan Settings**, and **Accessibility Test Overview**.
![Run Overview Section](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Section_Run_Results.png)

3. You can also check test suite results inside test machine by clicking on the **Test Machine** of your choice.
![Test Suite in Test Machine](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtsitm.png)
3. Click **View Runs** to view all runs, **More Settings** to access additional settings, and **View Report** to check the **Accessibility Testing Report**.
![View Overlay Components](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Overlay_Comps.png)

---

## **Exporting the Test Reports**

Here is a quick GIF demonstrating how to view test reports in Testsigma.
![Reports](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/DrillDownReports.gif)
1. Click the **Export** icon and select the desired download format from the dropdown menu.
![Export Run Results](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Export_Run_Results.png)

---
2. For **PDF format**, the **Export PDF** dialog appears. Select the appropriate options and click **Export**.
![Export PDF](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Export_Run_Results_In_PDF.png)

## **Additional Actions on Results**

There are multiple actions that you can take on these results. These actions are:
3. For **JUnit format**, the report is downloaded instantly as an **XML file**.

1. **Rerun**: You can initiate reruns of the test cases by clicking on Rerrun.
![Rerun](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrrerun.png)
4. For **XLSX format**, a downloadable link of the report is sent to your email.

2. **Export Reports**: You can download the report in Junit and MS Excel Sheet at test case/test suite/test machine levels by clicking on the **Export** icon.
![Export Icon](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrreicon.png)
---

3. **Test Run Details**: You can see the test run details below the result overview.
![Test Run Details](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtrdetails.png)
## **Available Shortcuts in Run Results**

### **For Mac:**
- **Option + C**: View results at the test case level
- **Option + S**: View results at the test suite level
- **Option + M**: View results at the test machine level
- **Option + H**: Open/close run history
- **Option + F**: Open filters
- **Option + O**: Open/close run overview

### **For Windows:**
- **Alt + C**: View results at the test case level
- **Alt + S**: View results at the test suite level
- **Alt + M**: View results at the test machine level
- **Alt + H**: Open/close run history
- **Alt + F**: Open filters
- **Alt + O**: Open/close run overview

---
Loading
Loading