diff --git a/src/pages/docs/debugging/debugging-localdevices-web.md b/src/pages/docs/debugging/debugging-localdevices-web.md index 306809bb..ad849175 100644 --- a/src/pages/docs/debugging/debugging-localdevices-web.md +++ b/src/pages/docs/debugging/debugging-localdevices-web.md @@ -49,7 +49,7 @@ Before you begin debugging in Testsigma, ensure the following: 1. Navigate to the test case you want to debug. 2. On the test case details page, click the **Launch Debugger** button in the top right corner of the screen. -3. On the Run in Debug Overlay, ensure the **Test Lab** is set to **Local Device** and that the **Agent** is in **Active Status** for debugging. +3. On the Run in Debug Overlay, default the **Test Lab** is set to **Local Device** and that the **Agent** is in **Active Status** for debugging. 4. To execute the test steps until a failed step, enable the **Run till failed step toggle** and set a debug point by selecting the test step from the dropdown. If this toggle is disabled, the test case will execute from start to end. 5. Click **Debug now** to execute the test case. ![click launch debugger in test case details page](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/launch_debugger_in_web_application.gif) diff --git a/src/pages/docs/debugging/launch-the-debugger-in-the-same-window.md b/src/pages/docs/debugging/launch-the-debugger-in-the-same-window.md index b3feb3b2..8f482a88 100644 --- a/src/pages/docs/debugging/launch-the-debugger-in-the-same-window.md +++ b/src/pages/docs/debugging/launch-the-debugger-in-the-same-window.md @@ -1,6 +1,6 @@ --- title: "Launch Debugger in the Same Window" -order: 10.4 +order: 10.5 page_id: "Launch debugger" metadesc: "This article discusses in detail step-by-step guide on how to run tests in debugger mode in the same window as well as new window in Testsigma application." search_keyword: "" diff --git a/src/pages/docs/debugging/results-on-local-devices.md b/src/pages/docs/debugging/results-on-local-devices.md index 1a1be88b..0984055a 100644 --- a/src/pages/docs/debugging/results-on-local-devices.md +++ b/src/pages/docs/debugging/results-on-local-devices.md @@ -1,85 +1,109 @@ --- -title: "Debug Results on Local Devices" -order: 10.3 -page_id: "Debug Results on Local Devices" -metadesc: "Debug issues in complex test cases with enhanced techniques. This article discusses in detail step-by-step guide on how to debug results on local devices in Testsigma." +title: "Debugging Mobile Applications on Local Devices" +pagetitle: "Debugging Web Application Results on Local Devices" +metadesc: "Learn how to debug test cases on local devices for web applications, using detailed steps and troubleshooting tips to ensure accurate and efficient testing." +noindex: false +order: 10.4 +page_id: "debugging-results-local-devices-mobile-applications" search_keyword: "" warning: false contextual_links: - type: section name: "Contents" - type: link - name: "Pre-requisites" - url: "#pre-requisites" + name: "Prerequisites" + url: "#prerequisites" - type: link - name: "Steps" - url: "#steps" + name: "Running Debugging on Android or iOS Devices" + url: "#running-debugging-on-android-or-i0s-devices" +- type: link + name: "Adding and Managing Debug Points on Mobile Devices" + url: "#adding-and-managing-debug-points-on-mobile-devices" +- type: link + name: "Interacting with Debug Points during Mobile Debugging" + url: "#interacting-with-debug-points-during-mobile-debugging" +- type: link + name: "Inspecting Test Execution Details on Mobile" + url: "#inspecting-test-execution-details-on-mobile" +- type: link + name: "Identifying and Fixing Errors during Mobile Test Case Execution" + url: "#identifying-and-fixing-errors-during-mobile-test-case-execution" +- type: link + name: "Troubleshooting Element Not Found Error on Mobile Devices" + url: "#troubleshooting-element-not-found-error-on-mobile-devices" --- --- -There might be times when you face an issue during your test execution, and you need more than screenshots, videos, or logs to see where the error was. With Testsigma, now you can debug any such issues on your local devices via step-wise debugging. In this document we will discuss how to do that. +Debugging mobile applications on local devices with Testsigma allows you to inspect and resolve errors by running test cases step-by-step. This interactive debugging feature lets you pause at specific points, inspect behavior, and analyze resources like step results, metadata, and screenshots, ensuring accurate problem identification and efficient resolution. --- ## **Prerequisites** -1. A mobile app test case to debug. To know more on how to create a test case for mobile, check [here](https://testsigma.com/docs/test-cases/manage/add-edit-delete/) -2. A local mobile device configured to run with Testsigma. Here are two links that would help: - 1. [Set Up Android Local Devices](https://testsigma.com/docs/agent/connect-android-local-devices/) - 2. [Set Up iOS Local Devices](https://testsigma.com/docs/agent/connect-ios-local-devices/) +Before starting the debugging process, ensure the following: +- A local mobile device with [Android](https://testsigma.com/docs/agent/connect-android-local-devices/) or [iOS](https://testsigma.com/docs/agent/connect-ios-local-devices/) applications configured with Testsigma. The Testsigma Agent is active on your local device. +- Debugging must be performed on a local device. +- If your test case uses a test data profile, only one test data profile can be selected for debugging. --- -## **Steps** - -1. Go to the test case that you want to debug on your local device. -2. On the test case details page, click on the downward arrow beside the run button on the top right of the page. For your reference, the downward arrow is also highlighted in the screenshot below: - -![The drop down to run a test case in debug mode.](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/debugging/results-on-local-devices/run-debug-mode-dropdown-button.png) - -3. You will see two options in the drop-down that appears, choose the option **Run In Debug Mode**. +## **Running Debugging on Android or iOS Devices** -4. The **Ad-hoc Run** layover will appear as also shown in the screenshot below. +1. Navigate to the test case you want to debug. +2. On the test case details page, click the **Launch Debugger** button in the top-right corner of the screen. +3. In the **Run in Debug Mode** overlay, ensure the following: + - Default the **Test Lab** is set to **Local Device** + - Select the **Test Machine** Device from the dropdown menu to choose the device for the debug session. + - Enable the **Run till Failed** step toggle to execute the test steps until a specific step fails. If this toggle is disabled, the test case will execute from start to end. + - Set a debug point by selecting a test step from the dropdown menu. + - Select **External Path** and provide the application's publicly accessible URL or **Uploaded Apps** and select the application from uploaded apps in Testsigma. +4. Click **Debug Now** to start the debugging process. The debugger screen will open, showing the test steps, related information, and the device screen. -![The drop down to run a test case in debug mode.](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/debugging/results-on-local-devices/ad-hoc-run-layover-debug-mode.png) - -5. On this layover, make sure the **Test Lab** is selected as **Local Devices**. Select the Test Device by choosing the **Test Machine** and **Device Name**. - -6. If you want the test cases to execute till the step that fails, then you have to enable toggle **Run till failed step**. +--- -Or, if you want the test cases to execute till a particular step then you can choose that step from the dropdown with label **Run till step**. +## **Adding and Managing Debug Points on Mobile Devices** -If you enable the toggle **Run till failed step** and also choose a particular step under **Run till step** then execution will run till the condition is met first. For example, if a failure is encountered first it will run till the step that fails, and vice versa. +Debug points are specific locations in your test steps where you want the execution to pause or start. To add and manage debug points: -If you enabled toggle **Run till failed step** then the debug point will appear at the failed step during test execution, if any. And, if you selected a step under 'Run till step', then a debug point will appear for that step. +1. In the Testsigma Debugger overlay, hover over the test step where you want to place a debug point. +2. Click the **Place Debug Point** icon to add a debug point. +3. To add multiple debug points, repeat the process for other test steps. +4. Execution will pause when it reaches an enabled debug point, allowing you to observe the behavior of the test case at that specific step. -7. Configure the section Use Path/Use Uploads according to your setup. +--- -8. Once you have filled the required fields, click on the button **Run Now** on the bottom right of the **Ad-hoc Run** layover. +## **Interacting with Debug Points during Mobile Debugging** -9. Now the debugger screen will open where you will see the test steps, related information and the device screen. The device screen will display the current screen being under test. The screen will appear something like the screenshot below: +When the execution pauses at a debug point, you can perform the following actions: +- **Resume Execution**: Continue the test execution from the paused point. +- **Run This Step**: Execute the current step immediately. +- **Step Over:** Skip the current step and move to the next one. -![The debug screen for debugging a test case for a mobile test case.](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/debugging/results-on-local-devices/debugger-screen-mobile.png) +--- -10. On this screen you would be able to see the test execution. The enabled debug point will be displayed like the red icon highlighted in the screenshot below. +## *Inspecting Test Execution Details on Mobile** -![Enabled debug point for a test step.](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/debugging/results-on-local-devices/enabled-debug-point.png) +- **Add Source**: When you reach a debug point, the **Add Source** section provides information about the test step's source code, allowing you to understand the context and behavior of the code being executed. +- **Selected Element**: In the **Selected Element** section, view details about the UI elements involved in the test step. This includes properties and attributes that are crucial for debugging element-related issues. +- **Step Result**: The **Step Result** section shows the outcome of the test step, including pass/fail status and any associated error messages. This information helps diagnose issues and verify whether the test step has behaved as expected. -[[info | **NOTE**:]] -|On this screen, the debug point can be enabled for any test step by hovering on the test step and clicking on the corresponding debug point that appears. When a debug point is enabled for a test step, the test execution will pause on reaching that step. +--- -11. When the test execution pauses on any of these steps, you can check the related details under the section **Add Source/Selected Element/Step Result**. In addition, you see below options on the test step: - 1. **Resume**: To resume the test execution from that step. - 2. **Rerun**: To execute the selected step again. - 3. **Step Over**: To start debugging the next step. If you click on the “step over” button on a debug step, you will see that the debug point will appear for the next step. The execution control would move to the next step too. +## **Identifying and Fixing Errors during Mobile Test Case Execution** +During test case execution, you can visually follow each step. If an error occurs or a debug point is reached: +- Testsigma will highlight the problematic step. +- Use the provided resources, including Add Source, Selected Element, and Step Result, to analyze the issue and determine the necessary fixes. +--- +## **Troubleshooting Element Not Found Error on Mobile Devices** +A common issue is the **Element Not Found** error, where an element visible on the device is reported as not found. To address this: -The test step actions are highlighted in the screenshot below for your reference: +1. View the message for the test step. If the element is not found, click **Element** to view its properties. +2. Analyze the issue, update the element properties if needed, then click **Options** and select **Resume Execution** from the previously paused step. -![The test step actions for debug](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/debugging/results-on-local-devices/debug-test-step-actions.png) +--- ---- \ No newline at end of file diff --git a/src/pages/docs/reports/runs/custom-pdf-report.md b/src/pages/docs/reports/runs/custom-pdf-report.md new file mode 100644 index 00000000..ce6755eb --- /dev/null +++ b/src/pages/docs/reports/runs/custom-pdf-report.md @@ -0,0 +1,241 @@ +--- +title: "Custom PDF Report Generation" +pagetitle: "Custom PDF Report Generation: Step-by-Step Guide" +metadesc: "Learn how to generate custom PDF reports with detailed steps, prerequisites, and preferences for personalized reports in this comprehensive guide." +noindex: false +order: 13.28 +page_id: "custom-pdf-report-generation" +search_keyword: "" +warning: false +contextual_links: +- type: section + name: "Contents" +- type: link + name: "Prerequisites" + url: "#prerequisites" +- type: link + name: "Retrieving Result IDs" + url: "#retrieving-result-ids" +- type: link + name: "Executing the JAR File" + url: "#executing-the-jar-file" +- type: link + name: "Using Preferences to Generate Custom PDF Report" + url: "#using-preferences-to-generate-custom-pdf-report" +- type: link + name: "Preferences for Custom Report Generation" + url: "#preferences-for-custom-report-generation" +--- + +--- + +Custom PDF Report Generator in Testsigma allows you to create detailed and customized reports according to your needs. This feature lets you generate reports for test cases, suites, or machines without restrictions. You can also adjust your reports using different preferences, making it a strong tool for in-depth analysis and presentation. + +--- + +## **Prerequisites** + +Before you begin, ensure you have the following: + +- An [API Key ](https://testsigma.com/docs/reports/runs/filter-custom-reports/) from your Testsigma account. +- An HTML template for the PDF report. +- Java 17 or a newer version installed on your system. +- The JAR file from Testsigma Customer Support. + +--- + +## **Retrieving Result IDs** + +To retrieve the Run ID from a run result: + +1. Navigate to the **Run Result** from which you want to retrieve the Run ID. +2. For **test plan** level reports, use the **Run ID** from the RunResult page or **URL endpoint**. ![retrieving runid for test plan level](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/testplan_run_id.cpr.gif) +3. For Test Machine, Test Suite, or Test Case level reports: + - Right-click on the page and select **Inspect** to open Developer Tools. + - Go to the **Network** tab. + - In the Testsigma application, select the test case, suite, or test machine in the Run Result details page to get the response. + - In the Developer Tools, under the **Network** tab, find the response results in the name section and click **Preview**. + - Unhide the **Content** and copy the required **ID**. ![retrieving result ids](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/case-machine-suitelevel-id-cpr.gif) + +--- + +## **Executing the JAR File** + +Run the Custom Report Generator with the required inputs to create a PDF report. This step converts your HTML template into a PDF based on your preferences. To execute the JAR file, follow these steps: + +1. Open Terminal or Command Prompt and navigate to the folder containing the JAR file. +2. Run the following command, replacing the placeholders with your actual values: +```bash +java -jar custom_pdf_generator-0.0.1-SNAPSHOT.jar \ + --config.apiKey=YOUR-API-KEY \ + --config.plan.runId=YOUR-PLAN-RUN-ID \ + --config.id=YOUR-ID \ + --config.preference.resultLevel=YOUR-PREFERENCE-RESULT-LEVEL \ + --config.template.location=/path/to/your/template.html \ + --config.pdf.directory=/path/to/save/report.pdf +``` + +
+Replace the placeholders with your actual values: + +- **YOUR-API-KEY**: Your API key for authentication with Testsigma. +- **YOUR-PLAN-RUN-ID**: The run ID of your test plan. +- **YOUR-ID**: The specific ID for Test Case, Test Suite, or Test Machine level reports. +- **YOUR-PREFERENCE-RESULT-LEVEL**: Specifies the detail level of the report (CASE, MACHINE, SUITE). +- **/path/to/your/template.html**: Path to your HTML template file. +- **/path/to/save/report.pdf**: Location where the generated PDF report will be saved. + +--- + +## **Using Preferences to Generate Custom PDF Report** + +Customize your report content and structure using various preferences: +```bash +java -jar custom_pdf_generator-0.0.1-SNAPSHOT.jar \ + --config.apiKey=YOUR-API-KEY \ + --config.plan.runId=YOUR-PLAN-RUN-ID \ + --config.id=YOUR-ID \ + --config.preference.resultLevel=YOUR-PREFERENCE-RESULT-LEVEL \ + --config.preference.step=(YOUR-PREFERENCE-STEP) \ + --config.preference.screenshot=(YOUR-PREFERENCE-SCREENSHOT) \ + --config.preference.visualDifference=(YOUR-PREFERENCE-VISUAL-DIFFERENCE) \ + --config.preference.summaryFields=(YOUR-PREFERENCE-CASE-DETAILS-HEADERS) \ + --config.template.location=/path/to/your/template.html \ + --config.pdf.directory=/path/to/save/report.pdf +``` + +
+Replace the placeholders with your actual values: + +- **YOUR-API-KEY**: Your API key for authentication with Testsigma. +- **YOUR-PLAN-RUN-ID**: The run ID of your test plan. +- **YOUR-ID**: The specific ID for Test Case, Test Suite, or Test Machine level reports. +- **YOUR-PREFERENCE-RESULT-LEVEL**: Specifies the detail level of the report (CASE, MACHINE, SUITE). +- **YOUR-PREFERENCE-STEP**:Defines the steps to include in the report. Possible values: + - **PASSED**: Only include passed steps. + - **FAILED**: Only include failed steps. + - **EXECUTED**: Only include executed steps. + - **NOT_EXECUTED**: Only include steps that were not executed. + - **ALL**: Include all steps. + - **NONE**: Exclude all steps. +- **YOUR-PREFERENCE-SCREENSHOT**: Defines which screenshots to include. Possible values: + - **PASSED**: Only include screenshots for passed tests. + - **FAILED**: Only include screenshots for failed tests. + - **ALL**: Include all screenshots. + - **NONE**: Exclude all screenshots. +- **YOUR-PREFERENCE-VISUAL-DIFFERENCE**: Defines which visual screenshots to include. Possible values: + - **PASSED**: Only include visual screenshots for passed tests. + - **FAILED**: Only include visual screenshots for failed tests. + - **ALL**: Include all visual screenshots. + - **NONE**: Exclude all visual screenshots. +- **YOUR-PREFERENCE-CASE-DETAILS-HEADERS**: Specifies which headers to include in case details. Example values: + - **testCaseName** + - **testSuiteName** +- **/path/to/your/template.html**: Path to your HTML template file. +- **/path/to/save/report.pdf**: Location where the generated PDF report will be saved. + +--- + +## **Preferences for Custom Report Generation** + +List of configurable preferences to tailor your PDF reports according to your needs. + + + + + + + Custom Report Preferences + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PreferenceDescriptionAllowed ValuesInput Type
config.preference.resultLevelDefines the level of detail for the reportPLAN, MACHINE, SUITE, CASESingle value
config.preference.stepSpecifies which steps to include in the reportPASSED, FAILED, EXECUTED, NOT_EXECUTED, ALL, NONESingle value
config.preference.screenshotIndicates which screenshots to includePASSED, FAILED, ALL, NONESingle value
config.preference.visualDifferenceSpecifies which visual screenshots to includePASSED, FAILED, ALL, NONESingle value
config.preference.summaryFieldsFields to display in the report summaryname, executedBy, environment, testPlanName, testDeviceName, testSuiteName, result, buildNo, runId, screenshotCapturedFor, screenshotModeMulti-value (comma-separated)
config.preference.caseListColumnsColumns to show in the test case listETF, testSuite, testMachine, assignee, reviewerMulti-value (comma-separated)
config.preference.caseDetailsHeadersColumns to show in case details headerstestCaseName, testSuiteNameMulti-value (comma-separated)
config.preference.stepListColumnsColumns to show in the step listreasonForFailure, testDataDetails, duration, apiResponseUrlMulti-value (comma-separated)
+ + + +--- + +