Skip to content

Commit

Permalink
Merge pull request #554 from testsigmahq/dev
Browse files Browse the repository at this point in the history
Updated Run Results docs as per updated UI
  • Loading branch information
bharathk08 authored Dec 11, 2024
2 parents e231955 + 18cf57d commit 31188c0
Show file tree
Hide file tree
Showing 9 changed files with 170 additions and 161 deletions.
7 changes: 4 additions & 3 deletions src/left-nav-title.json
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@
"/docs/reports/runs/rerun/": "ReRun"
},
"drill-down-reports": {
"/docs/reports/runs/drill-down-reports/": "Drill-down Reports"
"/docs/reports/runs/drill-down-reports/": "Test Plan Run Results"
},
"test-machine-reports": {
"/docs/reports/runs/test-machine-reports/": "Test Machine Level Reports"
Expand All @@ -370,7 +370,7 @@
"/docs/reports/runs/test-case-reports/": "Test Case Reports"
},
"filter-custom-reports": {
"/docs/reports/runs/filter-custom-reports": "Filters & Custom Reports"
"/docs/reports/runs/filter-custom-reports": "Filters on Run Results"
},
"export-report-junit": {
"/docs/reports/runs/export-report-junit/": "Export Reports - JUnit"
Expand Down Expand Up @@ -690,5 +690,6 @@
"generate-tdp": {"/docs/genai-capabilities/generate-tdp/": "Generate TDP Using Copilot"},
"smtp-configuration": {"/docs/configuration/smtp-configuration/": "SMTP Configuration"},
"generate-wda-file": {"/docs/configuration/generate-wda-file/": "Create WDA File" },
"environments": {"/docs/api/environments/": "Manage Environments"}
"environments": {"/docs/api/environments/": "Manage Environments"},
"runs": {"/docs/reports/runs/": "Results & Reports"}
}
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-report-pdf.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ In Testsigma, you can effortlessly export PDF reports with screenshots for your

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-report-xls.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ Effortlessly generate detailed Excel reports for your test runs in Testsigma. Th

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/reports/export-reports/export-reports-junit.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ You can conveniently export your test plan reports in XML format, specifically i

---

## **Prerequisites**

Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
> ## **Prerequisites**
>
> Before exporting reports, you must understand specific concepts, such as creating [Test Plans](https://testsigma.com/docs/test-management/test-plans/overview/), conducting [Test Runs](https://testsigma.com/docs/reports/runs/overview/), and having access to export test reports.
[[info | **NOTE**:]]
| Please contact us at **[email protected]** or use the **Instant Chat** option to enable this exclusive feature provided by **Testsigma Enterprise**.
Expand Down
22 changes: 11 additions & 11 deletions src/pages/docs/reports/runs/custom-pdf-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,17 @@ Custom PDF Report Generator in Testsigma allows you to create detailed and custo

---

## **Prerequisites**

Before you begin, ensure you have the following:

- An [API Key ](https://testsigma.com/docs/reports/runs/filter-custom-reports/) from your Testsigma account.

- **Java 17** or a newer version installed on your system.

- You can request the **JAR file** from the Customer Support team and download it before executing a Custom PDF Report Generation.

- An **HTML template** for the PDF report. [Click here](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Custom_Report_Template.txt) to download the HTML template.
> ## **Prerequisites**
>
> Before you begin, ensure you have the following:
>
> - An [API Key ](https://testsigma.com/docs/reports/runs/filter-custom-reports/) from your Testsigma account.
>
> - **Java 17** or a newer version installed on your system.
>
> - You can request the **JAR file** from the Customer Support team and download it before executing a Custom PDF Report Generation.
>
> - An **HTML template** for the PDF report. [Click here](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Custom_Report_Template.txt) to download the HTML template.
---

Expand Down
171 changes: 110 additions & 61 deletions src/pages/docs/reports/runs/drill-down-reports.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Drill Down Reports in Testsigma"
metadesc: "Drill-down reports let you access data at all layers. Learn how to use drill-down reports in the test execution results of automated tests in Testsigma"
title: "Test Plan Run Results in Testsigma"
metadesc: "View and download reports from the Run Results page in Testsigma | This page provides detailed results at the test cases, suites, and machines levels."
noindex: false
order: 13.23
page_id: "Drill Down Reports in Testsigma"
page_id: "Test Plan Run Results"
warning: false
contextual_links:
- type: section
Expand All @@ -12,117 +12,166 @@ contextual_links:
name: "Prerequisites"
url: "#prerequisites"
- type: link
name: "How to Use Drill-down Reports in Testsigma"
url: "#how-to-use-drill-down-reports-in-testsigma"
name: "Steps to View Test Suite Results"
url: "#steps-to-view-test-suite-results"
- type: link
name: "Steps to Generate Test Case Reports"
url: "#steps-to-generate-test-case-reports"
name: "Steps to View Test Case Results"
url: "#steps-to-view-test-case-results"
- type: link
name: "Steps to Generate Test Suite Reports"
url: "#steps-to-generate-test-suite-reports"
name: "Steps to View Test Machine Results"
url: "#steps-to-view-test-machine-results"
- type: link
name: "Steps to Generate Test Machine Reports"
url: "#steps-to-generate-test-machine-reports"
name: "Test Runs in Run Results"
url: "#test-runs-in-run-results"
- type: link
name: "Additional Actions on Results"
url: "#additional-actions-on-results"
name: "Overview of Tests in Run Results"
url: "#overview-of-tests-in-run-results"
- type: link
name: "Exporting the Test Reports"
url: "#exporting-the-test-reports"
- type: link
name: "Available Shortcuts in Run Results"
url: "#available-shortcuts-in-run-results"
---

---

The users will have better satisfaction when the application does not have bugs, technical glitches, performance issues, or usability issues. When these issues increase, the quality decreases, which results in bad UX. That's when drill-down reports come in handy.
In Testsigma, you can view and download reports from the Run Results page. This page provides results at the test cases, test suites, and test machine level and also helps you check test runs, view the run overview, and see the count of passed/failed tests along with the reasons for failure. This article provides an overview of the Run Results page in Testsigma.


In Testsigma, you can view/download reports at multiple levels and find specific details, like the number of tests failed and the reason for failures or machines causing more problems. You can assess the application's quality and address issues from this data.
---

> ## **Prerequisites**
>
> Before you begin, make sure you have created and executed a [test plan](https://testsigma.com/docs/test-management/test-plans/overview/).
---

## **Prerequisites**
## **Steps to View Test Suite Results**

- You should know how to [create a test case](https://testsigma.com/docs/test-cases/manage/add-edit-delete/#creating-a-test-case).
1. From the left navigation bar, go to **Run Results** and click on the test plan for which you want to check the results.
![Test Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Test_Suites.png)

- You should know how to [create a test suite](https://testsigma.com/docs/test-management/test-plans/overview/#steps-to-create-a-test-plan).
2. By default, you’ll see the results at test suite level.
![Results at Test Suite Level](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Default_Page.png)

- You should know how to [create a test plan](https://testsigma.com/docs/test-management/test-suites/overview/#creating-a-test-suit).
3. You can expand the test suite to check test case results inside the suite.
![Expand Test Suite](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Expan_Test_Suite_Run_Results.png)

4. The symbols shown with the arrow in the image below indicate the number of test suites running in parallel and the number of test cases running in parallel within each test suite.
![Parallel Settings Indicators](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Parallel_Settings_In_Test_Suites.png)

---

## **How to Use Drill-down Reports in Testsigma:**

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
## **Steps to View Test Case Results**

1. On the **Run Results** page, select **Test Cases** from the dropdown menu.
![Test Cases](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Results_Test_Cases.png)

2. From **Test Plan Result** page, you can view **Test Case** reports.
![Test Case Report](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtcr.png)
2. The page displays all test cases from all test suites.
![Test Cases in all Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Cases_In_All_Suites.png)

3. You can hover over a failed test case to view a brief description of the reason for failure.
![Failure Dialog Box](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Failure_Dialog_Test_Cases.png)

[[info | **NOTE**:]]
| You can also view reports at test suite and test machine level by clicking on **Test Suite** and **Test Machine**.
| ![Reoprts](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtstm.png)
4. Click a test case to view its detailed results.

3. You can choose the different runs from the dropdown to see how the results have changed for different builds.
![Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrrabuilds.png)
5. On the **Test Case Results** page, you can view all the test steps along with the **Element Information, Test Data, Settings, Metadata**, and **Step Details**.
![Detailed Result of a Test Case](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Detailed_Test_Case_Result.png)

6. Click **More details** to access additional information.
![More Details](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/More_Details_On_Test_Case_Result.png)

---

## **Steps to View Test Machine Results**

## **Steps to Generate Test Case Reports**
1. On the **Run Results** page, select **Test Machines** from the dropdown menu.
![Test Machine Run Results](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Machine_Run_Results.png)

2. The page displays all test machines included in the test plan.
![All Test Machines](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/All_Test_Machines_Run_Results.png)

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Case** to view the results grouped by **Test Cases**.
![Test Case Report](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtcr.png)
[[info | **NOTE**:]]
| To view the test cases part of the test machine, navigate to the **Test Suites** view and apply a **Test Machine** filter. This provides a view of all test cases in the selected test suite and test machine.

3. Click a test machine to expand and view the test suites it contains.
![Expand Machine](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Expand_Test_Machine_Run_Result.png)

---

## **Test Runs in Run Results**

1. From the **Test Runs** panel, select a different run to view its corresponding test run results.
![Test Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Test_Runs_In_Run_Results.png)

## **Steps to Generate Test Suite Reports**
2. Click **Rerun** in the top-right corner to rerun the test plan by selecting the appropriate rerun options.
![Rerun](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ReRun_from_Run_Results.png)

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
[[info | **NOTE**:]]
| 1. Each run ID has a maximum rerun limit of 10. We recommend using reruns wisely.
| ![Rerun Limit](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Rerun_Limits_Run_Results.png)
|
| 2. Once you click **Rerun**, you'll see the following options.
| - **All Test Cases:** This will rerun all the test cases in the selected run.
| - **All Failed Test Cases:** This will rerun all the failed test cases in the selected run.
| - **Select Cases for Re-Run:** This will allow you to select the test cases you want to rerun.
|
| 3. Click **Start execution** to rerun the test plan.

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Suite** to view the results grouped by **Test Suites**.
![Test Suite](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtestsuite.png)

3. You can also check test case results inside test suite by clicking on the **Test suite** of your choice.
![Test Case in Test Suites](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtstclr.png)
1. To compare the current rerun result with the previous result, click **Compare Run**.
![Compare Test Runs](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Compare_Runs_In_Run_Results.png)

2. This opens the **Rerun** overlay, where you can view a detailed comparison at all levels.
![Rerun Overlay](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Compare_Runs_Overlay.png)

---

## **Steps to Generate Test Machine Reports**
## **Overview of Tests in Run Results**

1. Navigate to **Run Results** page and click on a **Test Plan Result**.
![Test Plan Result](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrnav.png)
1. The **Run Overview** section of the **Run Results** page is dynamic and provides a summary of tests at all levels for the selected run.
![Run Overview](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Run_Results.png)

2. Results are available in **Test Case**, **Test Suite**, and **Test Machine** level. Click on **Test Machine** to view the results grouped by **Test Machines**.
![Test Machine Reports](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtmr.png)
2. This section displays the **Last Run, Test Plan Settings**, and **Accessibility Test Overview**.
![Run Overview Section](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Section_Run_Results.png)

3. You can also check test suite results inside test machine by clicking on the **Test Machine** of your choice.
![Test Suite in Test Machine](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtsitm.png)
3. Click **View Runs** to view all runs, **More Settings** to access additional settings, and **View Report** to check the **Accessibility Testing Report**.
![View Overlay Components](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Run_Overview_Overlay_Comps.png)

---

## **Exporting the Test Reports**

Here is a quick GIF demonstrating how to view test reports in Testsigma.
![Reports](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/DrillDownReports.gif)
1. Click the **Export** icon and select the desired download format from the dropdown menu.
![Export Run Results](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Export_Run_Results.png)

---
2. For **PDF format**, the **Export PDF** dialog appears. Select the appropriate options and click **Export**.
![Export PDF](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/Export_Run_Results_In_PDF.png)

## **Additional Actions on Results**

There are multiple actions that you can take on these results. These actions are:
3. For **JUnit format**, the report is downloaded instantly as an **XML file**.

1. **Rerun**: You can initiate reruns of the test cases by clicking on Rerrun.
![Rerun](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrrerun.png)
4. For **XLSX format**, a downloadable link of the report is sent to your email.

2. **Export Reports**: You can download the report in Junit and MS Excel Sheet at test case/test suite/test machine levels by clicking on the **Export** icon.
![Export Icon](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrreicon.png)
---

3. **Test Run Details**: You can see the test run details below the result overview.
![Test Run Details](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ddrrtrdetails.png)
## **Available Shortcuts in Run Results**

### **For Mac:**
- **Option + C**: View results at the test case level
- **Option + S**: View results at the test suite level
- **Option + M**: View results at the test machine level
- **Option + H**: Open/close run history
- **Option + F**: Open filters
- **Option + O**: Open/close run overview

### **For Windows:**
- **Alt + C**: View results at the test case level
- **Alt + S**: View results at the test suite level
- **Alt + M**: View results at the test machine level
- **Alt + H**: Open/close run history
- **Alt + F**: Open filters
- **Alt + O**: Open/close run overview

---
Loading

0 comments on commit 31188c0

Please sign in to comment.