From 492499b8e4bdfe6d2ef521d38e171bef6ee0123b Mon Sep 17 00:00:00 2001 From: Sangith Ravichandran <119392848+sangithravichandran@users.noreply.github.com> Date: Tue, 2 Jul 2024 13:55:26 +0530 Subject: [PATCH 1/2] Added Agent Log Level Configuration section (#437) --- src/pages/docs/agent/troubleshooting/logs.md | 81 ++++++++++++-------- 1 file changed, 48 insertions(+), 33 deletions(-) diff --git a/src/pages/docs/agent/troubleshooting/logs.md b/src/pages/docs/agent/troubleshooting/logs.md index 4b126c2c..b29a9209 100644 --- a/src/pages/docs/agent/troubleshooting/logs.md +++ b/src/pages/docs/agent/troubleshooting/logs.md @@ -1,55 +1,71 @@ --- title: "Agent Logs" -metadesc: "How to fetach the logs files from the Testsigma Agent to troubleshoot the errors. These files help you resolve execution issues quickly in Testsigma App." +page_title: "Agent Logs: Configuration, Sending, and Debugging Guide" +metadesc: "Learn to configure agent log levels, send logs automatically or manually, and debug errors efficiently to resolve execution issues quickly." noindex: false order: 11.92 -page_id: "Fetching Agent logs" +page_id: "testsigma-agent-logs" search_keyword: "" warning: false contextual_links: - type: section name: "Contents" +- type: link + name: "Agent Log Level Configuration" + url: "#agent-log-level-configuration" - type: link name: "Send the Logs Automatically" url: "#send-the-logs-automatically" - type: link - name: "Send the Logs manually" + name: "Send the Logs Manually" url: "#send-the-logs-manually" +- type: link + name: "Debugging Agent Log Errors" + url: "#debugging-agent-log-errors" --- --- -To execute Testsigma tests on your local machines, a small utility program named Testsigma agent needs to be on each local machine. In this article, we will discuss how to send the log files of the Testsigma agent. Providing these files to the Testsigma Support Team will help us resolve execution issues quickly. - +Testsigma Agent is a critical utility program that must be installed on each local machine to execute Testsigma tests. Proper log management is essential for debugging and maintaining the health of these automation agents. This guide covers everything from configuring log levels to automatically or manually sending log files to the Testsigma Support Team. This document explains how to configure agent log levels, send logs automatically and manually, and debug errors. --- -## **Send the Logs Automatically** +## **Agent Log Level Configuration** -1. From the dashboard, go to **Agents**. +Agent log-level configuration in Testsigma is important for debugging and keeping your automation agents healthy. By default, the INFO level is selected. This setting shows in the command line interface (CLI) and is written to the agent log files on your local device. When you report an agent issue, these logs are automatically sent to the Testsigma support team. -2. Hover over the agent of which you want to send logs, and click **Report Agent**. +### **Change Agent Level Configuration** -3. In the warning dialog, provide a description of the problem you're facing with the agent and click **Report**. +1. Click on the **Agents** icon in the left navigation bar. Open an agent from the Agent list page. +2. Click on the **Agent Settings** icon in the right navigation bar. Choose the agent log level configuration for CLI and local files from the log level options. ![Choose Agent Log Level](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/go_to_agent_log_b.gif) + - **ERROR**: Logs only critical issues that cause the agent to fail. Only error messages will appear in the CLI and be saved in the local agent log files. + - **WARN**: Logs warnings and errors to spot potential issues early. Warning and error messages will appear in the CLI and be saved in the local agent log files. + - **INFO**: The default level that logs general information, warnings, and errors for regular monitoring. Info, warning, and error messages will appear in the CLI and be saved in the local agent log files. + - **DEBUG**: Logs detailed information for debugging. Debug info, warnings, and errors will appear in the CLI and be saved in the local agent log files. + - **TRACE**: Logs detailed traces of the execution flow for deep debugging. Trace messages, debug info, warnings, and errors will appear in the CLI and be saved in the local agent log files. + - **ALL**: Logs all messages, providing the most detailed logging. All messages (error, warning, info, debug, and trace) will appear in the CLI and be saved in the local agent log files. ![agent log level configuration](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/agent_log_level_configuration.png) +3. After selecting the log-level configuration, **restart** the agent for the changes to take effect. ![choose log level and retsart agent](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/select_loglevel_restart.gif) -4. The logs will be reported to Testsigma support automatically. +[[info | NOTE:]] +| Always restart the agent after changing the log-level configuration to apply the new settings. +--- -Here’s a quick GIF demonstrating how to send the logs automatically. +## **Send the Logs Automatically** -![Send Logs Automatically](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/agentlogsauto.gif) +1. From the dashboard, go to **Agents**. +2. Hover over the agent for which you want to send logs, and click **Report Agent**. +3. In the warning dialog, provide a description of the problem you're facing with the agent and click **Report**. +4. The logs will be reported to Testsigma support automatically. +Here’s a quick GIF demonstrating how to send the logs automatically. ![send logs automatically](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/agentlogsauto.gif) --- -## **Send the Logs manually** - -You can find the Testsigma agent logs folder for your machine as mentioned below: +## **Send the Logs Manually** -**Location of Testsigma Agent Configuration/Logs** - -The Testsigma Configuration/Log files are located at the following locations depending on the Operating System used: +You can find the Testsigma agent logs folder for your machine at the following locations, depending on the operating system used: | Operating System | Location of Agent directory | | :--- | :--- | @@ -58,13 +74,13 @@ The Testsigma Configuration/Log files are located at the following locations dep | Mac | $HOME/Library/Application Support/Testsigma/Agent/ | - The **%userprofile%** points to the **C:\Users\< username > folder where < username >** is replaced by your Windows login username. -- $HOME in Linux/Mac also points to the user's home folder +- **$HOME** in Linux/Mac also points to the user's home folder -There would be a logs folder in the above directory. Inside the **logs** directory, you can find a number of files such as agent.log, Testengine.log e.t.c. +Inside the above directory, you will find a **logs** folder containing several files such as agent.log, Testengine.log, etc. -1. Please zip the **Agent** folder mentioned above. +1. Zip the **Agent** folder mentioned above. -2. Once you have the agent logs ZIP file, hover the agent of which you want to send logs and click **Report Agent**. +2. Hover over the agent for which you want to send logs and click **Report Agent**. 3. Click **Upload manually**. This will open a warning dialog. @@ -72,21 +88,20 @@ There would be a logs folder in the above directory. Inside the **logs** directo Here’s a quick GIF demonstrating how to send the logs manually. -![Agent Logs Manually](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/agentlogsmanual.gif) +![agent logs manually](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/agentlogsmanual.gif) --- -**Debugging Errors** - -For a technical user, the logs will serve as a great tool for debugging the issues themselves. However, if you need an extra hand, we are always there for you. Just drop in a mail to **support@testsigma.com** describing the issue and with the above-mentioned log files attached. Remember, adding as much detail as possible will help us resolve the issue in the shortest time. +## **Debugging Agent Log Errors** +For technical users, the logs serve as a valuable tool for debugging issues. If you need extra assistance, contact **support@testsigma.com**, providing a description of the issue along with the log files mentioned above. Including as much detail as possible will help resolve the issue quickly. +Please include the following details in your email: +- **Project Name** +- **Application Name** +- **Test Case Name** +- **Execution Name** +- **Execution Type**: Hybrid Web/Hybrid Mobile (if Hybrid Mobile Execution, attach Appium Logs, which can be downloaded from the Appium Server. Contact Support for more details). -Please mention the following details in the mail: - -* **Project Name**, **Application Name**, **Test Case Name**, and **Execution Name**. -* **Execution Type**: Hybrid Web/ Hybrid Mobile. If it is Hybrid Mobile Execution, attaching the Appium Logs too will help. You can download the Appium logs from the Appium Server. Contact Support for more details. - - ---- \ No newline at end of file +--- From 5df5545e6e2a68d9f174b8bfa3768eba472b3b84 Mon Sep 17 00:00:00 2001 From: Bharath Krishna <118433150+bharathk08@users.noreply.github.com> Date: Thu, 4 Jul 2024 19:13:46 +0530 Subject: [PATCH 2/2] Added a new doc for Testsigma Private Grid (#438) Added a new doc for test executions on Testsigma Private Grid. --- .../runs/executing-tests-in-private-grid.md | 182 ++++++++++++++++++ .../docs/runs/test-plans-on-local-devices.md | 2 +- .../docs/salesforce-testing/connected-app.md | 2 +- .../metadata-connections.md | 4 +- 4 files changed, 186 insertions(+), 4 deletions(-) create mode 100644 src/pages/docs/runs/executing-tests-in-private-grid.md diff --git a/src/pages/docs/runs/executing-tests-in-private-grid.md b/src/pages/docs/runs/executing-tests-in-private-grid.md new file mode 100644 index 00000000..3d1d5970 --- /dev/null +++ b/src/pages/docs/runs/executing-tests-in-private-grid.md @@ -0,0 +1,182 @@ +--- +title: "Run Tests on Private Grid" +metadesc: "Testsigma Private Grid helps run parallel tests across multiple browsers, OS, and machines. This article discusses how to setup and execute tests on Private Grid" +noindex: false +order: 9.41 +page_id: "Test Plan Local Executions" +warning: false +contextual_links: +- type: section + name: "Contents" +- type: link + name: "Prerequisites" + url: "#prerequisites" +- type: link + name: "Create Dynamic Agent Through API" + url: "#create-dynamic-agent-through-api" +- type: link + name: "Start Dynamic Agent Using jwtApiKey" + url: "#start-dynamic-agent-using-jwtapikey" +- type: link + name: "Start the Private Grid Hub & Web Nodes" + url: "#start-the-private-grid-hub--web-nodes" +- type: link + name: "Execute Tests in Private Grid using REST API" + url: "#execute-tests-in-private-grid-using-rest-api" +- type: link + name: "Execute Tests in Private Grid from Application" + url: "#execute-tests-in-private-grid-from-application" +--- + + +--- + +Testsigma Private Grid helps run parallel tests across multiple browsers, operating systems, and machines. This can boost the execution time of even the largest test suites and save minutes, hours, or days. + +![Testsigma Private Grid](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/TestsigmaHub.png) + +**Hub**
+The Hub is the main controller in Private Grid. Once it receives the test requests, it distributes them to different machines called Nodes. The Hub knows what each Node can do. It evaluates the test requirements against the capabilities of the available Nodes to select the best match and forwards the tests to the selected Nodes. + + + +**Nodes**
+Like Test Machines in Testsigma, Nodes are the machines that run the tests in Private Grid. They can have multiple browsers and operating systems. A session is created when the Node receives a test request from the client. Any commands sent to this Node will then be executed, and the response will be returned to the client. Nodes can handle multiple test sessions simultaneously, depending on their configuration and resource capacity. + +This article discusses executing tests in Testsigma Private Grid. + + +--- + +> ## **Prerequisites** +> +> - Testsigma Agent +> - Core Java (OpenJDK 18) +> - Testsigma Private Grid Folder +> - Ngrok Setup + + +--- + +## **Create Dynamic Agent Through API** +Use the following Endpoint to create Dynamic Agents with Private Grid configuration in Testsigma using REST API. + +|**Request Type**|**POST**| +|---|---| +|**Endpoint**| https://app.testsigma.com/api/v1/agents | +|**Authorization**| Bearer ****
Same as the Testsigma API key mentioned above.| +|**Request Body Type (JSON)**| Raw | +|**Payload**|
{
"name": "<Unique Agent Name>",
"agentType": "TRANSIENT",
"externalConfig": {
"url": "http://localhost:4444",
"authenticationEnabled": true,
"userName": "root",
"password": "root",
"type": "SELENIUM",
"browserList": ["Chrome", "Safari"]
}
}
|
+|**Response Body Type (JSON)**|
{
"id": 9,
"uniqueId": "a3cceab7-5bc9-41d1-a905-e54e0a23c677",
"jwtApiKey": "<Unique JWT Token>",
"agentVersion": null,
"createdById": 2,
"updatedById": 2,
"createdDate": 1719222650559,
"updatedDate": 1719222650559,
"system": {
"title": "Dynamic Agent",
"id": 21,
"protocol": "http",
"port": null,
"httpsPort": null,
"ipAddress": "127.0.0.1",
"hostName": null,
"osType": null,
"osVersion": null,
"systemType": 4
},
"browserList": [],
"currentAgentVersion": "5.9.0",
"status": "REGISTERED",
"isActive": false
}
|
+
+Here’s  a quick GIF demonstrating how to create a Dynamic Agent through API with the above mentioned endpoint.
+
+![Create Dynamic Agent](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/CreateDynamicAgent.gif)
+
+---
+
+## **Start Dynamic Agent Using jwtApiKey**
+
+Open the terminal in the **Testsigma Agent directory** and use the following command to start the **Dynamic Agent**:
+
+> **For Mac/Linux**
+>
+>
<agent_dir>./start.sh --TS_ACTIVATION_KEY=<JWT_KEY>
+ + +> **For Windows** +> +>
<agent_dir>./start.bat --TS_ACTIVATION_KEY=<JWT_KEY>
+ +Where <JWT_KEY> is **jwtApiKey** from the above response. + +Here’s a quick GIF demonstrating how to start Dynamic Agent using jwtApiKey with the above command. + +![Up Testsigma Agent](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/UpDynamicAgent.gif) + +--- + +## **Start the Private Grid Hub & Web Nodes** + +To start the **Private Grid Hub**, open the **Terminal** at **Private Grid** folder and use the following command: + +
java -jar selenium-server-4.13.0.jar hub
+ +Once the **Private Grid** starts, open http://localhost:4444 + +To start the Web Nodes, open the new tab on the same Terminal and use the following command: + +
java -jar selenium-server-4.13.0.jar node --config node.json --selenium-manager true
+ +Once the command runs, the registered **Nodes** will be added to the **Private Grid**. + +Here’s a quick GIF demonstrating the steps to start the Private Grid Hub and add Web Nodes. +![Private Grid Setup](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/PrivateHub.gif) + +--- + +## **Execute Tests in Private Grid using REST API** + +Use the following **Endpoint** to execute the Testsigma’s test plan in **Private Grid** using REST API. + +|**Request Type**|**POST**| +|---|---| +|**Endpoint**| https://app.testsigma.com/api/v1/execution_results | +|**Authorization**| Bearer ****
Same as the Testsigma API key mentioned above.| +|**Request Body Type (JSON)**| Raw | +|**Payload**|
 {
"executionId" : <Test Plan Exe ID>,
"testDevices" : [
{
"title" : "Dynamic Machine",
"targetMachine" : 27,
"browser" : "CHROME",
"executionLabType" : "Hybrid",
"suiteIds" : [<Suite IDs>],
"isHeadless" : false,
"runTestSuitesInParallel": true,
"runTestCasesInParallel": true,
"createSessionAtCaseLevel": true
}
]
}
|
+|**Sample Response Body (JSON)**|
{
"id": 581,
"execution": null,
"executionId": 346,
"startTime": 1719400027825,
"endTime": null,
"duration": null,
"result": "QUEUED",
"status": "STATUS\_CREATED",
"message": "Test Plan execution message",
"executedBy": 29,
"buildNo": null,
"environmentId": null,
"totalCount": null,
"failedCount": null,
"passedCount": null,
"stoppedCount": null,
"notExecutedCount": null,
"queuedCount": null,
"runningCount": null,
"visualTestResult": null,
"environment": null,
"childResult": null,
"reRunParentId": null,
"triggeredType": "API",
"totalRunningCount": 0,
"executionDetails": {
"page\_timeout": 30,
"element\_timeout": 30,
"recovery\_action": "Run\_Next\_Testcase",
"on\_aborted\_action": "Reuse\_Session",
"screenshot\_option": "ALL\_TYPES",
"screenshot\_mode": null,
"group\_prerequisite\_fail": "Abort",
"test\_case\_prerequisite\_fail": "Abort",
"test\_step\_prerequisite\_fail": "Run\_Next\_Testcase",
"global\_param\_name": null
},
"totalQueuedCount": 0,
"mobileInspectionId": null,
"environmentResultsDTO": [],
"executionResultConfig": null,
"isReRunEnabled": false,
"consolidatedResult": "QUEUED",
"consolidatedStatus": "STATUS\_CREATED",
"resultType": "ORIGINAL",
"consolidatedDuration": null,
"runTestCasesInParallel": false,
"runTestSuitesInParallel": false,
"scheduledId": null,
"testPlanHookResults": null,
"reRunType": null,
"consolidatedPlanTotalCount": null,
"consolidatedPlanFailedCount": null,
"consolidatedPlanPassedCount": null,
"consolidatedPlanStoppedCount": null,
"consolidatedPlanNotExecutedCount": null,
"consolidatedPlanQueuedCount": null,
"consolidatedPlanRunningCount": null,
"testDevicesNames": null,
"latestResult": null,
"testPlanResultMetric": null,
"activeExecutionResultCount": 0
}
|
+
+Here’s  a quick GIF demonstrating how to execute tests through a Dynamic Agent through API with the above mentioned endpoint.
+![Execute Tests using API](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/executetestswithgrid.gif)
+
+---
+
+## **Execute Tests in Private Grid from Application**
+
+### **a. Integrating Testsigma with Private Grid**
+
+1. From the left navigation bar, go to **Settings > Integrations** and choose the **Test Lab** tab.
+
+2. Enable the toggle on the **Private Grid** widget. A popup screen will display the details for the **Private Grid** lab.
+
+3. On **Private Grid** details, 
+    - If there is authentication enabled in ngrok platform, select **Authentication Enabled** and enter **Username** and **Password**. 
+    - Enter the **Private Hub URL** and select applications to use with your **Private Grid** framework.
+
+4. Click on **Add Details**.
+
+
+ +
+ + ℹ️Fetching the Private Hub URL: +
+Follow the steps below to set up ngrok on your device to fetch the Private Hub URL: +
    +
      +
    1. Visit ngork website, create an account, go to Getting Started > Setup & Installation.
    2. +
    3. Follow the instructions on the Setup & Installation page to set up ngrok.
    4. +
    5. Once the installation is complete, enter the command ngrok http 4444 on the terminal.
    6. +
    7. Copy the URL as shown in the image below and use this to integrate Testsigma with Private Grid.
    8. + Trulli +
    +
+
+ +
+ +### **b. Execute Tests in Private Grid** + +1. For test case execution, select **Private Grid** as **Test Lab** on **Ad-hoc Run** overlay. + + Here’s a quick GIF demonstrating how to execute a test case in Private Grid. + ![Test Case Execution](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/tcexeinPG.gif) + +2. For test plan execution, add **Private Grid** while creating test machine profiles for execution. You need to go to **Add Test Suites & Link Machine Profiles > Select test machine profiles > Add Machine** and add **Private Grid** as **Test Lab**. + + Here’s a quick GIF demonstrating how to execute a test plan in Private Grid. + + ![Test Plan Execution](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/ExecuteTestinPGUI.gif) + +--- \ No newline at end of file diff --git a/src/pages/docs/runs/test-plans-on-local-devices.md b/src/pages/docs/runs/test-plans-on-local-devices.md index 93d48a3d..c8401d4c 100644 --- a/src/pages/docs/runs/test-plans-on-local-devices.md +++ b/src/pages/docs/runs/test-plans-on-local-devices.md @@ -2,7 +2,7 @@ title: "Test Plan Local Executions" metadesc: "Local execution is preferred for testing apps that are only available locally, for security purposes. How to perform test plan executions on local devices in Testsigma." noindex: false -order: 9.41 +order: 9.42 page_id: "Test Plan Local Executions" warning: false contextual_links: diff --git a/src/pages/docs/salesforce-testing/connected-app.md b/src/pages/docs/salesforce-testing/connected-app.md index 262cd822..20ff29e2 100644 --- a/src/pages/docs/salesforce-testing/connected-app.md +++ b/src/pages/docs/salesforce-testing/connected-app.md @@ -52,7 +52,7 @@ Creating a connected app is important to link external applications and the Sale 5. In the **API** section, select the **Enable OAuth Settings** checkbox to set **Callback URL** and **Selected OAuth Scopes**. The **Selected OAuth Scopes** are mandatory in the **Connected App** to connect & download the metadata. - - Use this Callback URL: https://salesforce-callback.testsigma.com/authorize + - Use this Callback URL: https://salesforce-callback.testsigma.com/authorize/ ![API Details](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/sfcaapiinfo.png) diff --git a/src/pages/docs/salesforce-testing/metadata-connections.md b/src/pages/docs/salesforce-testing/metadata-connections.md index cbea6d41..d203edb9 100644 --- a/src/pages/docs/salesforce-testing/metadata-connections.md +++ b/src/pages/docs/salesforce-testing/metadata-connections.md @@ -67,7 +67,7 @@ There are two ways to Authenticate:
  • User name & Password: For User name & Password, enter Username, Password, Security token, Consumer key, and Consumer secret.
  • OAuth: For OAuth, enter Username, Consumer key, and Consumer secret. -
  • Callback URL: https://salesforce-callback.testsigma.com/authorize
  • +
  • Callback URL: Use the below mentioned Callback URL.

    https://salesforce-callback.testsigma.com/authorize/
@@ -86,7 +86,7 @@ There are two ways to Authenticate: - Wait until the sync is complete. This will take some time. ![Wait](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/sfmdsipg.png) -1. Once Metadata sync is successful, you can create test cases by clicking **Create Test Case**. +4. Once Metadata sync is successful, you can create test cases by clicking **Create Test Case**. ![Create Tests](https://s3.amazonaws.com/static-docs.testsigma.com/new_images/projects/applications/sfmdctc.png) ---