Skip to content

Commit

Permalink
Extension of the ReportPortal plugin using API
Browse files Browse the repository at this point in the history
    * option `--merge` to create suite per plan and merge them all in one launch
      (stored launch uuid in run of the first plan)
    * option '--attributes' for additional attributes,
      but mainly for assignment of attributes to the launch with merged plans
    * option `--uuid` to append new plans to an existing launch
    * store launch uuid as rp_uuid per merged run, and as launch_uuid per each plan,
      store launch_url per each plan
    * rewritten environment variables to the uniform form TMT_PLUGIN_REPORT_REPORTPORTAL_${option}
    * prepared defect type locator for implementation of continuous update (idle)
    * prepared rerun for implementation of launch update (retry)
    * mapping according to options --launch-per-plan and --suite-per-plan
    * uploading to existing launch/suite with options --upload-to-launch LAUNCH_ID, --upload-to-suite SUITE_ID
    * option --launch-description and preparation for --launch-attributes (for suite-per-plan mapping)
    * trial option --launch-rerun
    * trial option --defect-type
    * functional --suite-per-plan option that uploads all plans into a launch;
      + reporting common atributes from all plans, closing the launch after last plan
    * additional upload of tests/suites into a existing launch
    * functional idle report and additional report within run id
    * functional upload to launch
    * debug based on option combinations
    * fixed the functonality to report idle tests and additional results
  • Loading branch information
4N0body5 authored Feb 6, 2024
1 parent 97b84ca commit afe1a56
Show file tree
Hide file tree
Showing 3 changed files with 445 additions and 129 deletions.
46 changes: 37 additions & 9 deletions spec/plans/report.fmf
Original file line number Diff line number Diff line change
Expand Up @@ -104,19 +104,47 @@ description:
web page, filter them via context attributes and get links
to detailed test output and other test information.
description:
Fill json with test results and other fmf data per each plan,
and send it to a Report Portal instance via its API.
Provide test results and fmf data per each plan,
and send it to a Report Portal instance via its API
with token, url and project name given.
example:
- |
# Set environment variables with the server url and token
export TMT_REPORT_REPORTPORTAL_URL=<url-to-RP-instance>
export TMT_REPORT_REPORTPORTAL_TOKEN=<token-from-RP-profile>
# Optionally set environment variables according to TMT_PLUGIN_REPORT_REPORTPORTAL_${OPTION}
export TMT_PLUGIN_REPORT_REPORTPORTAL_URL=${url-to-RP-instance}
export TMT_PLUGIN_REPORT_REPORTPORTAL_TOKEN=${token-from-RP-profile}
- |
# Enable ReportPortal report from the command line
# Enable ReportPortal report from the command line depending on the use case:

## Simple upload with all project, url endpoint and user token passed in command line
tmt run --all report --how reportportal --project=baseosqe --url="https://reportportal.xxx.com" --token="abc...789"

## Simple upload with url and token exported in environment variable
tmt run --all report --how reportportal --project=baseosqe
tmt run --all report --how reportportal --project=baseosqe --exclude-variables="^(TMT|PACKIT|TESTING_FARM).*"
tmt run --all report --how reportportal --project=baseosqe --launch=test_plan
tmt run --all report --how reportportal --project=baseosqe --url=... --token=...

## Upload with project name in fmf data, filtering out parameters (environemnt variables) that tend to be unique and break the history aggregation
tmt run --all report --how reportportal --exclude-variables="^(TMT|PACKIT|TESTING_FARM).*"

## Upload all plans as suites into one ReportPortal launch
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-description="..."

## Rerun the launch with suite structure for the test results to be uploaded into the latest launch with the same name as a new 'Retry' tab (mapping based on unique paths)
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-rerun

## Rerun the tmt run and append the new result logs under the previous one uploaded in ReportPortal (precise mapping)
tmt run --id run-012 --all report --how reportportal --again

## Additional upload of new suites into given launch with suite structure
tmt run --all report --how reportportal --suite-per-plan --upload-to-launch=4321

## Additional upload of new tests into given launch with non-suite structure
tmt run --all report --how reportportal --launch-per-plan --upload-to-launch=1234

## Additional upload of new tests into given suite
tmt run --all report --how reportportal --upload-to-suite=123456

## Upload Idle tests, then execute it and add result logs into prepared empty tests
tmt run discover report --how reportportal --defect-type=Idle
tmt run --last --all report --how reportportal --again
- |
# Use ReportPortal as the default report for given plan
report:
Expand Down
21 changes: 21 additions & 0 deletions tmt/schemas/report/reportportal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,27 @@ properties:
launch:
type: string

launch-description:
type: string

launch-per-plan:
type: boolean

suite-per-plan:
type: boolean

upload-to-launch:
type: string

upload-to-suite:
type: string

launch-rerun:
type: boolean

defect_type:
type: string

exclude-variables:
type: string

Expand Down
Loading

0 comments on commit afe1a56

Please sign in to comment.