Code coverage measures used and unused code lines, statements, branches, etc. Depending on the programming language this
+is measured by instrumenting the code/binary and running the program, it’s test cases or simulating the code. In
+generate code coverage is a measure of test coverage. Unused code is not (yet) covered by tests.
+
The code coverage metric in percent is a ratio of used code versus all possibly usable code. A coverage of <100%
+indicates unused code. This can be dead code (unreachable) or untested code (⇒ needs more test cases).
When installed via PIP, the command line program pyedaa-reports is registered in the Python installation’s
+Scripts directory. Usually this path is listed in PATH, thus this program is globally available after
+installation.
+
The program is self-describing. Use pyedaa-reports without parameters or pyedaa-reportshelp to see all
+available common options and commands. Each command has then it’s own help page for command specific options, which can
+be listed by calling pyedaa-reports<cmd>-h or pyedaa-reportshelp<cmd>. The pyedaa-reports’s version and
+license information is shown by calling pyedaa-reportsversion.
Unit Testing / Coverage / Type Checking (Optional)
+
Additional Python packages needed for testing, code coverage collection and static type checking. These packages are
+only needed for developers or on a CI server, thus sub-dependencies are not evaluated further.
+
Manually Installing Test Requirements
+
Use the tests/requirements.txt file to install all dependencies via pip3. The file will recursively install
+the mandatory dependencies too.
Additional Python packages needed for documentation generation. These packages are only needed for developers or on a
+CI server, thus sub-dependencies are not evaluated further.
+
Manually Installing Documentation Requirements
+
Use the doc/requirements.txt file to install all dependencies via pip3. The file will recursively install
+the mandatory dependencies too.
Additional Python packages needed for installation package generation. These packages are only needed for developers or
+on a CI server, thus sub-dependencies are not evaluated further.
+
Manually Installing Packaging Requirements
+
Use the build/requirements.txt file to install all dependencies via pip3. The file will recursively
+install the mandatory dependencies too.
Additional Python packages needed for publishing the generated installation package to e.g, PyPI or any equivalent
+services. These packages are only needed for maintainers or on a CI server, thus sub-dependencies are not evaluated
+further.
+
Manually Installing Publishing Requirements
+
Use the dist/requirements.txt file to install all dependencies via pip3. The file will recursively
+install the mandatory dependencies too.
Creative Commons Corporation (“Creative Commons”) is not a law firm and does not
+provide legal services or legal advice. Distribution of Creative Commons public
+licenses does not create a lawyer-client or other relationship. Creative Commons
+makes its licenses and related information available on an “as-is” basis.
+Creative Commons gives no warranties regarding its licenses, any material
+licensed under their terms and conditions, or any related information. Creative
+Commons disclaims all liability for damages resulting from their use to the
+fullest extent possible.
+
+
Creative Commons Attribution 4.0 International Public License
+
By exercising the Licensed Rights (defined below), You accept and agree to be
+bound by the terms and conditions of this Creative Commons Attribution 4.0
+International Public License (“Public License”). To the extent this Public
+License may be interpreted as a contract, You are granted the Licensed Rights
+in consideration of Your acceptance of these terms and conditions, and the
+Licensor grants You such rights in consideration of benefits the Licensor
+receives from making the Licensed Material available under these terms and
+conditions.
Adapted Material means material subject to Copyright and Similar
+Rights that is derived from or based upon the Licensed Material and in
+which the Licensed Material is translated, altered, arranged, transformed, or
+otherwise modified in a manner requiring permission under the Copyright and
+Similar Rights held by the Licensor. For purposes of this Public License,
+where the Licensed Material is a musical work, performance, or sound
+recording, Adapted Material is always produced where the Licensed Material
+is synched in timed relation with a moving image.
+
Adapter’s License means the license You apply to Your Copyright and
+Similar Rights in Your contributions to Adapted Material in accordance with
+the terms and conditions of this Public License.
+
Copyright and Similar Rights means copyright and/or similar rights
+closely related to copyright including, without limitation, performance,
+broadcast, sound recording, and Sui Generis Database Rights, without regard
+to how the rights are labeled or categorized. For purposes of this Public
+License, the rights specified in Section 2(b)(1)-(2) are not Copyright and
+Similar Rights.
+
Effective Technological Measures means those measures that, in the
+absence of proper authority, may not be circumvented under laws fulfilling
+obligations under Article 11 of the WIPO Copyright Treaty adopted on
+December 20, 1996, and/or similar international agreements.
+
Exceptions and Limitations means fair use, fair dealing, and/or any
+other exception or limitation to Copyright and Similar Rights that applies to
+Your use of the Licensed Material.
+
Licensed Material means the artistic or literary work, database, or
+other material to which the Licensor applied this Public License.
+
Licensed Rights means the rights granted to You subject to the terms
+and conditions of this Public License, which are limited to all Copyright and
+Similar Rights that apply to Your use of the Licensed Material and that the
+Licensor has authority to license.
+
Licensor means the individual(s) or entity(ies) granting rights under
+this Public License.
+
Share means to provide material to the public by any means or process
+that requires permission under the Licensed Rights, such as reproduction,
+public display, public performance, distribution, dissemination,
+communication, or importation, and to make material available to the public
+including in ways that members of the public may access the material from a
+place and at a time individually chosen by them.
+
Sui Generis Database Rights means rights other than copyright
+resulting from Directive 96/9/EC of the European Parliament and of the
+Council of 11 March 1996 on the legal protection of databases, as amended
+and/or succeeded, as well as other essentially equivalent rights anywhere
+in the world.
+
You means the individual or entity exercising the Licensed Rights
+under this Public License. Your has a corresponding meaning.
Subject to the terms and conditions of this Public License, the Licensor
+hereby grants You a worldwide, royalty-free, non-sublicensable,
+non-exclusive, irrevocable license to exercise the Licensed Rights in the
+Licensed Material to:
+
+
+
reproduce and Share the Licensed Material, in whole or in part; and
+
produce, reproduce, and Share Adapted Material.
+
+
+
+
Exceptions and Limitations. For the avoidance of doubt, where
+Exceptions and Limitations apply to Your use, this Public License does not
+apply, and You do not need to comply with its terms and conditions.
+
Term. The term of this Public License is specified in Section 6(a).
+
Media and formats; technical modifications allowed. The Licensor
+authorizes You to exercise the Licensed Rights in all media and formats
+whether now known or hereafter created, and to make technical
+modifications necessary to do so. The Licensor waives and/or agrees not to
+assert any right or authority to forbid You from making technical
+modifications necessary to exercise the Licensed Rights, including
+technical modifications necessary to circumvent Effective Technological
+Measures. For purposes of this Public License, simply making modifications
+authorized by this Section 2(a)(4) never produces Adapted Material.
+
Downstream recipients.
+
+
+
Offer from the Licensor – Licensed Material. Every recipient of
+the Licensed Material automatically receives an offer from the
+Licensor to exercise the Licensed Rights under the terms and
+conditions of this Public License.
+
No downstream restrictions. You may not offer or impose any
+additional or different terms or conditions on, or apply any Effective
+Technological Measures to, the Licensed Material if doing so restricts
+exercise of the Licensed Rights by any recipient of the Licensed
+Material.
+
+
+
+
No endorsement. Nothing in this Public License constitutes or may
+be construed as permission to assert or imply that You are, or that Your
+use of the Licensed Material is, connected with, or sponsored, endorsed,
+or granted official status by, the Licensor or others designated to
+receive attribution as provided in Section 3(a)(1)(A)(i).
+
+
+
Other rights.
+
+
Moral rights, such as the right of integrity, are not licensed under this
+Public License, nor are publicity, privacy, and/or other similar
+personality rights; however, to the extent possible, the Licensor waives
+and/or agrees not to assert any such rights held by the Licensor to the
+limited extent necessary to allow You to exercise the Licensed Rights, but
+not otherwise.
+
Patent and trademark rights are not licensed under this Public License.
+
To the extent possible, the Licensor waives any right to collect royalties
+from You for the exercise of the Licensed Rights, whether directly or
+through a collecting society under any voluntary or waivable statutory or
+compulsory licensing scheme. In all other cases the Licensor expressly
+reserves any right to collect such royalties.
Your exercise of the Licensed Rights is expressly made subject to the following conditions.
+
+
Attribution.
+
+
If You Share the Licensed Material (including in modified form), You must:
+
+
+
retain the following if it is supplied by the Licensor with the
+Licensed Material:
+
+
+
+
identification of the creator(s) of the Licensed Material and any
+others designated to receive attribution, in any reasonable manner
+requested by the Licensor (including by pseudonym if designated);
+
a copyright notice;
+
a notice that refers to this Public License;
+
a notice that refers to the disclaimer of warranties;
+
a URI or hyperlink to the Licensed Material to the extent reasonably
+practicable;
+
+
+
+
indicate if You modified the Licensed Material and retain an
+indication of any previous modifications; and
+
indicate the Licensed Material is licensed under this Public License,
+and include the text of, or the URI or hyperlink to, this Public
+License.
+
+
+
+
You may satisfy the conditions in Section 3(a)(1) in any reasonable manner
+based on the medium, means, and context in which You Share the Licensed
+Material. For example, it may be reasonable to satisfy the conditions by
+providing a URI or hyperlink to a resource that includes the required
+information.
+
If requested by the Licensor, You must remove any of the information
+required by Section 3(a)(1)(A) to the extent reasonably practicable.
+
If You Share Adapted Material You produce, the Adapter’s License You apply
+must not prevent recipients of the Adapted Material from complying with
+this Public License.
Where the Licensed Rights include Sui Generis Database Rights that apply to Your
+use of the Licensed Material:
+
+
for the avoidance of doubt, Section 2(a)(1) grants You the right to extract,
+reuse, reproduce, and Share all or a substantial portion of the contents of
+the database;
+
if You include all or a substantial portion of the database contents in a
+database in which You have Sui Generis Database Rights, then the database
+in which You have Sui Generis Database Rights (but not its individual
+contents) is Adapted Material; and
+
You must comply with the conditions in Section 3(a) if You Share all or a
+substantial portion of the contents of the database.
+
+
For the avoidance of doubt, this Section 4 supplements and does not replace
+Your obligations under this Public License where the Licensed Rights include
+other Copyright and Similar Rights.
+
+
+
Section 5 – Disclaimer of Warranties and Limitation of Liability.
+
+
Unless otherwise separately undertaken by the Licensor, to the extent
+possible, the Licensor offers the Licensed Material as-is and as-available,
+and makes no representations or warranties of any kind concerning the
+Licensed Material, whether express, implied, statutory, or other. This
+includes, without limitation, warranties of title, merchantability,
+fitness for a particular purpose, non-infringement, absence of latent or
+other defects, accuracy, or the presence or absence of errors, whether or
+not known or discoverable. Where disclaimers of warranties are not allowed
+in full or in part, this disclaimer may not apply to You.
+
To the extent possible, in no event will the Licensor be liable to You
+on any legal theory (including, without limitation, negligence) or
+otherwise for any direct, special, indirect, incidental, consequential,
+punitive, exemplary, or other losses, costs, expenses, or damages arising
+out of this Public License or use of the Licensed Material, even if the
+Licensor has been advised of the possibility of such losses, costs, expenses,
+or damages. Where a limitation of liability is not allowed in full or in
+part, this limitation may not apply to You.
+
The disclaimer of warranties and limitation of liability provided above
+shall be interpreted in a manner that, to the extent possible, most
+closely approximates an absolute disclaimer and waiver of all liability.
This Public License applies for the term of the Copyright and Similar Rights
+licensed here. However, if You fail to comply with this Public License, then
+Your rights under this Public License terminate automatically.
+
Where Your right to use the Licensed Material has terminated under
+Section 6(a), it reinstates:
+
+
automatically as of the date the violation is cured, provided it is cured
+within 30 days of Your discovery of the violation; or
+
upon express reinstatement by the Licensor.
+
+
For the avoidance of doubt, this Section 6(b) does not affect any right the
+Licensor may have to seek remedies for Your violations of this Public License.
+
+
For the avoidance of doubt, the Licensor may also offer the Licensed Material
+under separate terms or conditions or stop distributing the Licensed Material
+at any time; however, doing so will not terminate this Public License.
+
Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
The Licensor shall not be bound by any additional or different terms or
+conditions communicated by You unless expressly agreed.
+
Any arrangements, understandings, or agreements regarding the Licensed
+Material not stated herein are separate from and independent of the terms
+and conditions of this Public License.
For the avoidance of doubt, this Public License does not, and shall not be
+interpreted to, reduce, limit, restrict, or impose conditions on any use of
+the Licensed Material that could lawfully be made without permission under
+this Public License.
+
To the extent possible, if any provision of this Public License is deemed
+unenforceable, it shall be automatically reformed to the minimum extent
+necessary to make it enforceable. If the provision cannot be reformed, it
+shall be severed from this Public License without affecting the
+enforceability of the remaining terms and conditions.
+
No term or condition of this Public License will be waived and no failure to
+comply consented to unless expressly agreed to by the Licensor.
+
Nothing in this Public License constitutes or may be interpreted as a
+limitation upon, or waiver of, any privileges and immunities that apply to
+the Licensor or You, including from the legal processes of any jurisdiction
+or authority.
+
+
+
Creative Commons is not a party to its public licenses. Notwithstanding,
+Creative Commons may elect to apply one of its public licenses to material it
+publishes and in those instances will be considered the “Licensor.” Except for
+the limited purpose of indicating that material is shared under a Creative
+Commons public license or as otherwise permitted by the Creative Commons
+policies published at creativecommons.org/policies,
+Creative Commons does not authorize the use of the trademark “Creative Commons”
+or any other trademark or logo of Creative Commons without its prior written
+consent including, without limitation, in connection with any unauthorized
+modifications to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For the
+avoidance of doubt, this paragraph does not form part of the public licenses.
Documentation coverage measures the presence of code documentation. It primarily counts for public language entities
+like publicly visible constants and variables, parameters, types, functions, methods, classes, modules, packages, etc.
+The documentation goal depends on the used coverage collection tool’s settings. E.g. usually, private language entities
+are not required to be documented.
+
The documentation coverage metric in percent is a ratio of documented language entity versus all documentation worthy
+langauge entities. A coverage of <100% indicates undocumented code.
A Test Entity is an entity in the test entity hierarchy. It can be a Test Case, Test Suite or
+Test Suite Summary. Some hierarchies (like JUnit) got extended by a Test Class.
The following instruction are using PIP (Package Installer for Python) as a package manager and PyPI (Python Package
+Index) as a source of Python packages.
For development and bug fixing it might be handy to create a local wheel package and also install it locally on the
+development machine. The following instructions will create a local wheel package (*.whl) and then use PIP to
+install it. As a user might have a pyEDAA.Reports installation from PyPI, it’s recommended to uninstall any previous
+pyEDAA.Reports packages. (This step is also needed if installing an updated local wheel file with same version number.
+PIP will not detect a new version and thus not overwrite/reinstall the updated package contents.)
cd<pyEDAA.Reports>
+
+# Package the code in a wheel (*.whl)
+python-mbuild--wheel
+
+# Uninstall the old package
+python-mpipuninstall-ypyEDAA.Reports
+
+# Install from wheel
+python-mpipinstall./dist/pyEDAA.Reports-0.1.0-py3-none-any.whl
+
+
+
+
+
+
cd <pyEDAA.Reports>
+
+# Package the code in a wheel (*.whl)
+py-mbuild--wheel
+
+# Uninstall the old package
+py-mpipuninstall-ypyEDAA.Reports
+
+# Install from wheel
+py-mpipinstall.\dist\pyEDAA.Reports-0.1.0-py3-none-any.whl
+
“License” shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
+
“Licensor” shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
+
“Legal Entity” shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that
+entity. For the purposes of this definition, “control” means (i) the power, direct or indirect, to cause the direction or management of such entity, whether
+by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
+
“You” (or “Your”) shall mean an individual or Legal Entity exercising permissions granted by this License.
+
“Source” form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and
+configuration files.
+
“Object” form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object
+code, generated documentation, and conversions to other media types.
+
“Work” shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is
+included in or attached to the work (an example is provided in the Appendix below).
+
“Derivative Works” shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions,
+annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works
+shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
+
“Contribution” shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative
+Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to
+submit on behalf of the copyright owner. For the purposes of this definition, “submitted” means any form of electronic, verbal, or written communication
+sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue
+tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is
+conspicuously marked or otherwise designated in writing by the copyright owner as “Not a Contribution.”
+
“Contributor” shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently
+incorporated within the Work.
Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such
+Derivative Works in Source or Object form.
Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such
+license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of
+their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim
+or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then
+any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form,
+provided that You meet the following conditions:
+
+
You must give any other recipients of the Work or Derivative Works a copy of this License; and
+
You must cause any modified files to carry prominent notices stating that You changed the files; and
+
You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source
+form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
+
If the Work includes a “NOTICE” text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the
+attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the
+following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the
+Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE
+file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute,
+alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
+
+
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or
+distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise
+complies with the conditions stated in this License.
Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and
+conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any
+separate license agreement you may have executed with Licensor regarding such Contributions.
This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable
+and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an “AS IS” BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT,
+MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and
+assume any risks associated with Your exercise of permissions under this License.
In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate
+and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or
+consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages
+for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been
+advised of the possibility of such damages.
While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other
+liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole
+responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
+
+
Appendix: How to apply the Apache License to your work
+
To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets “[]” replaced with your own identifying
+information. (Don’t include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or
+class name and description of purpose be included on the same “printed page” as the copyright notice for easier identification within third-party archives.
+
Copyright [yyyy] [name of copyright owner]
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
The unified data model for test entities (test summary, test suite, test case) implements a super-set of all (so far
+known) unit test result summary file formats. pyEDAA.Report’s data model is a structural and functional cleanup of the
+Ant JUnit data model. Naming has been cleaned up and missing features have been added.
+
As some of the JUnit XML dialects are too divergent from the original Ant + JUnit4 format, these dialects have an
+independent test entity inheritance hierarchy. Nonetheless, instances of each data format can be converted to and from
+the unified data model.
A test suite is a group of test cases and/or test suites. Test suites itself can be grouped by test
+suites. The test suite hierarchy’s root element is a test suite summary.
TestcaseStatus and TestsuiteStatus are
+flag enumerations to describe the overall status of a test case or test suite.
+
+
Unknown
tbd
+
+
Excluded
tbd
+
+
Skipped
tbd
+
+
Weak
tbd
+
+
Passed
tbd
+
+
Failed
tbd
+
+
Inverted
tbd
+
+
Warned
tbd
+
+
Errored
tbd
+
+
Failed
tbd
+
+
SetupError
tbd
+
+
TearDownError
tbd
+
+
Inconsistent
tbd
+
+
+
+
+
@export
+classTestcaseStatus(Flag):
+"""A flag enumeration describing the status of a test case."""
+ Unknown=0#: Testcase status is uninitialized and therefore unknown.
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Weak=4#: No assertions were recorded.
+ Passed=8#: A passed testcase, because all assertions were successful.
+ Failed=16#: A failed testcase due to at least one failed assertion.
+
+ Mask=Excluded|Skipped|Weak|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+ Inconsistent=32768#: Dataset is inconsistent
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError|Inconsistent
+
+
+
@export
+classTestsuiteStatus(Flag):
+"""A flag enumeration describing the status of a test suite."""
+ Unknown=0
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Empty=4#: No tests in suite
+ Passed=8#: Passed testcase, because all assertions succeeded
+ Failed=16#: Failed testcase due to failing assertions
+
+ Mask=Excluded|Skipped|Empty|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError
+
A Testcase is the leaf-element in the test entity hierarchy and describes an
+individual test run. Besides a test case status, it also contains statistics like the start time or the test
+duration. Test cases are grouped by test suites and need to be unique per parent test suite.
+
A test case (or its base classes) implements the following properties and methods:
The test case has a reference to it’s parent test suite in the hierarchy. By iterating parent references, the
+root element (test suite summary) be be found, which has no parent reference (None).
The test case has a name. This name must be unique per hierarchy parent, but can exist multiple times in the
+overall test hierarchy.
+
In case the data format uses hierarchical names like pyEDAA.Reports.CLI.Application, the name is split at
+the separator and multiple hierarchy levels (test suites) are created in the unified data model. To be able to
+recreate such an hierarchical name, TestsuiteKind is applied accordingly
+to test suite’s Kind field.
The test case stores a time when the individual test run was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test case has fields to capture the setup duration, test run duration and teardown duration. The sum of all
+durations is provided by total duration.
The setup duration is the time spend on setting up a test run. If the setup duration can’t be
+distinguished from the test’s runtime, set this value to None.
+
The test’s runtime without setup and teardown portions is captured by test duration. If the duration is
+unknown, set this value to None.
+
The teardown duration of a test run is the time spend on tearing down a test run. If the teardown
+duration can’t be distinguished from the test’s runtime, set this value to None.
+
The test case has a field total duration to sum up setup duration, test duration and teardown duration.
+If the duration is unknown, this value will be None.
The assertion count represents the overall number of assertions (checks) in a test case. It can be
+distinguished into passed assertions and failed assertions. If it can’t be distinguished, set
+passed and failed assertions to None.
A Testsuite is a grouping element in the test entity hierarchy and describes
+a group of test runs. Besides a list of test cases and a test suite status, it also contains statistics like the
+start time or the test duration for the group of tests. Test suites are grouped by other test suites or a test
+suite summary and need to be unique per parent test suite.
+
A test suite (or its base classes) implements the following properties and methods:
The test suite has a reference to it’s parent test entity in the hierarchy. By iterating parent references, the
+root element (test suite summary) be be found, which has no parent reference (None).
The test suite has a name. This name must be unique per hierarchy parent, but can exist multiple times in the
+overall test hierarchy.
+
In case the data format uses hierarchical names like pyEDAA.Reports.CLI.Application, the name is split at
+the separator and multiple hierarchy levels (test suites) are created in the unified data model. To be able to
+recreate such an hierarchical name, TestsuiteKind is applied accordingly
+to test suite’s Kind field.
The test suite stores a time when the first test run was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test suite has fields to capture the suite’s setup duration, test group run duration and suite’s
+teardown duration. The sum of all durations is provided by total duration.
The setup duration is the time spend on setting up a test suite. If the setup duration can’t be
+distinguished from the test group’s runtime, set this value to None.
+
The test group’s runtime without setup and teardown portions is captured by test duration. If the
+duration is unknown, set this value to None.
+
The teardown duration of a test suite is the time spend on tearing down a test suite. If the teardown
+duration can’t be distinguished from the test group’s runtime, set this value to None.
+
The test suite has a field total duration to sum up setup duration, test duration and teardown duration.
+If the duration is unknown, this value will be None.
A TestsuiteSummary is the root element in the test entity hierarchy and
+describes a group of test suites as well as overall statistics for the whole set of test cases. A test suite
+summary is derived for the same base-class as a test suite, thus they share almost all properties and methods.
+
A test suite summary (or its base classes) implements the following properties and methods:
The test suite summary stores a time when the first test runs was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test suite summary has fields to capture the suite summary’s setup duration, overall run duration and
+suite summary’s teardown duration. The sum of all durations is provided by total duration.
The setup duration is the time spend on setting up an overall test run. If the setup duration can’t be
+distinguished from the test’s runtimes, set this value to None.
+
The test suite summary’s runtime without setup and teardown portions is captured by test duration. If
+the duration is unknown, set this value to None.
+
The teardown duration of a test suite summary is the time spend on tearing down a test suite summary. If
+the teardown duration can’t be distinguished from the test’s runtimes, set this value to None.
+
The test suite summary has a field total duration to sum up setup duration, overall run duration and
+teardown duration. If the duration is unknown, this value will be None.
A JUnit XML test report summary file can be read by creating an instance of the Document
+class. Because JUnit has so many dialects, a derived subclass for the dialect might be required. By choosing the
+right Document class, also the XML schema for XML schema validation gets pre-selected.
Any JUnit dialect specific data model can be converted to the generic hierarchy of test entities.
+
+
Note
+
This conversion is identical for all derived dialects.
+
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Read from XML file
+xmlReport=Path("JUnit-Report.xml")
+try:
+ doc=Document(xmlReport,parse=True)
+exceptUnittestExceptionasex:
+ ...
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert to a tree
+rootNode=doc.ToTree()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+newDoc.Write()
+
A test suite summary can be converted to a document of any JUnit dialect. Internally a deep-copy is created to
+convert from a hierarchy of the unified test entities to a hierarchy of specific test entities (e.g. JUnit
+entities).
+
When the document was created, it can be written to disk.
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.AntJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.CTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.GoogleTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.PyTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/Unittesting/JUnitDataModel.html b/Unittesting/JUnitDataModel.html
new file mode 100644
index 00000000..7cd6dd4a
--- /dev/null
+++ b/Unittesting/JUnitDataModel.html
@@ -0,0 +1,621 @@
+
+
+
+
+
+
+
+
+ JUnit Data Model — pyEDAA.Reports 0.14.1 documentation
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
As the JUnit XML format was not well specified and no XML Schema Definition (XSD) was provided, many variants and
+dialects (and simplifications) were created by the various frameworks emitting JUnit XML files.
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.AntJUnit4importDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.CTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.GoogleTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.PyTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
Open Source VHDL Verification Methodology writes test results as YAML files for its
+internal data model storage. Some YAML files are written by the VHDL code of the verification framework, others are
+written by OSVVM-Scripts as a test runner.
pyEDAA.Reports provides a unified and generic unittest summary data model. The data model allows the description of
+testcases grouped in testsuites. Testsuites can be nested in other testsuites. The data model’s root element is a
+special testsuite called testsuite summary. It contains only testsuites, but no testcases.
+
The data model can be filled from various sources like Ant JUnit test reports or OSVVM testsuite summaries (more
+to be added). Many programming languages and/or unit testing frameworks support exporting results in the Ant JUnit
+format. See below for supported formats and their variations (dialects).
+
+
Attention
+
The so called JUnit XML format is the weakest file format and standard ever seen. At first was not created by JUnit
+(version 4). It was added by the built system Ant, but it’s not called Ant XML format nor Ant JUnit XML format. The
+latest JUnit 5 uses a completely different format called open test reporting. As
+JUnit is not the formats author, no file format documentation nor XML schema was provided. Also Ant isn’t providing
+any file format documentation or XML schema. Various Ant JUnit XML adopters have tried to reverse engineer a
+description and XML schemas, but unfortunately many are not even compatible to each other.
The unified data model for test entities (test summary, test suite, test case) implements a super-set of all (so far
+known) unit test result summary file formats. pyEDAA.Report’s data model is a structural and functional cleanup of the
+Ant JUnit data model. Naming has been cleaned up and missing features have been added.
+
As some of the JUnit XML dialects are too divergent from the original Ant + JUnit4 format, these dialects have an
+independent test entity inheritance hierarchy. Nonetheless, instances of each data format can be converted to and from
+the unified data model.
A test suite is a group of test cases and/or test suites. Test suites itself can be grouped by test
+suites. The test suite hierarchy’s root element is a test suite summary.
TestcaseStatus and TestsuiteStatus are
+flag enumerations to describe the overall status of a test case or test suite.
+
+
Unknown
tbd
+
+
Excluded
tbd
+
+
Skipped
tbd
+
+
Weak
tbd
+
+
Passed
tbd
+
+
Failed
tbd
+
+
Inverted
tbd
+
+
Warned
tbd
+
+
Errored
tbd
+
+
Failed
tbd
+
+
SetupError
tbd
+
+
TearDownError
tbd
+
+
Inconsistent
tbd
+
+
+
+
+
@export
+classTestcaseStatus(Flag):
+"""A flag enumeration describing the status of a test case."""
+ Unknown=0#: Testcase status is uninitialized and therefore unknown.
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Weak=4#: No assertions were recorded.
+ Passed=8#: A passed testcase, because all assertions were successful.
+ Failed=16#: A failed testcase due to at least one failed assertion.
+
+ Mask=Excluded|Skipped|Weak|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+ Inconsistent=32768#: Dataset is inconsistent
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError|Inconsistent
+
+
+
@export
+classTestsuiteStatus(Flag):
+"""A flag enumeration describing the status of a test suite."""
+ Unknown=0
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Empty=4#: No tests in suite
+ Passed=8#: Passed testcase, because all assertions succeeded
+ Failed=16#: Failed testcase due to failing assertions
+
+ Mask=Excluded|Skipped|Empty|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError
+
A Testcase is the leaf-element in the test entity hierarchy and describes an
+individual test run. Besides a test case status, it also contains statistics like the start time or the test
+duration. Test cases are grouped by test suites and need to be unique per parent test suite.
+
A test case (or its base classes) implements the following properties and methods:
The test case has a reference to it’s parent test suite in the hierarchy. By iterating parent references, the
+root element (test suite summary) be be found, which has no parent reference (None).
The test case has a name. This name must be unique per hierarchy parent, but can exist multiple times in the
+overall test hierarchy.
+
In case the data format uses hierarchical names like pyEDAA.Reports.CLI.Application, the name is split at
+the separator and multiple hierarchy levels (test suites) are created in the unified data model. To be able to
+recreate such an hierarchical name, TestsuiteKind is applied accordingly
+to test suite’s Kind field.
The test case stores a time when the individual test run was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test case has fields to capture the setup duration, test run duration and teardown duration. The sum of all
+durations is provided by total duration.
The setup duration is the time spend on setting up a test run. If the setup duration can’t be
+distinguished from the test’s runtime, set this value to None.
+
The test’s runtime without setup and teardown portions is captured by test duration. If the duration is
+unknown, set this value to None.
+
The teardown duration of a test run is the time spend on tearing down a test run. If the teardown
+duration can’t be distinguished from the test’s runtime, set this value to None.
+
The test case has a field total duration to sum up setup duration, test duration and teardown duration.
+If the duration is unknown, this value will be None.
The assertion count represents the overall number of assertions (checks) in a test case. It can be
+distinguished into passed assertions and failed assertions. If it can’t be distinguished, set
+passed and failed assertions to None.
A Testsuite is a grouping element in the test entity hierarchy and describes
+a group of test runs. Besides a list of test cases and a test suite status, it also contains statistics like the
+start time or the test duration for the group of tests. Test suites are grouped by other test suites or a test
+suite summary and need to be unique per parent test suite.
+
A test suite (or its base classes) implements the following properties and methods:
The test suite has a reference to it’s parent test entity in the hierarchy. By iterating parent references, the
+root element (test suite summary) be be found, which has no parent reference (None).
The test suite has a name. This name must be unique per hierarchy parent, but can exist multiple times in the
+overall test hierarchy.
+
In case the data format uses hierarchical names like pyEDAA.Reports.CLI.Application, the name is split at
+the separator and multiple hierarchy levels (test suites) are created in the unified data model. To be able to
+recreate such an hierarchical name, TestsuiteKind is applied accordingly
+to test suite’s Kind field.
The test suite stores a time when the first test run was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test suite has fields to capture the suite’s setup duration, test group run duration and suite’s
+teardown duration. The sum of all durations is provided by total duration.
The setup duration is the time spend on setting up a test suite. If the setup duration can’t be
+distinguished from the test group’s runtime, set this value to None.
+
The test group’s runtime without setup and teardown portions is captured by test duration. If the
+duration is unknown, set this value to None.
+
The teardown duration of a test suite is the time spend on tearing down a test suite. If the teardown
+duration can’t be distinguished from the test group’s runtime, set this value to None.
+
The test suite has a field total duration to sum up setup duration, test duration and teardown duration.
+If the duration is unknown, this value will be None.
A TestsuiteSummary is the root element in the test entity hierarchy and
+describes a group of test suites as well as overall statistics for the whole set of test cases. A test suite
+summary is derived for the same base-class as a test suite, thus they share almost all properties and methods.
+
A test suite summary (or its base classes) implements the following properties and methods:
The test suite summary stores a time when the first test runs was started. In combination with
+TotalDuration, the end time can be calculated. If the start time is
+unknown, set this value to None.
The test suite summary has fields to capture the suite summary’s setup duration, overall run duration and
+suite summary’s teardown duration. The sum of all durations is provided by total duration.
The setup duration is the time spend on setting up an overall test run. If the setup duration can’t be
+distinguished from the test’s runtimes, set this value to None.
+
The test suite summary’s runtime without setup and teardown portions is captured by test duration. If
+the duration is unknown, set this value to None.
+
The teardown duration of a test suite summary is the time spend on tearing down a test suite summary. If
+the teardown duration can’t be distinguished from the test’s runtimes, set this value to None.
+
The test suite summary has a field total duration to sum up setup duration, overall run duration and
+teardown duration. If the duration is unknown, this value will be None.
As the JUnit XML format was not well specified and no XML Schema Definition (XSD) was provided, many variants and
+dialects (and simplifications) were created by the various frameworks emitting JUnit XML files.
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.AntJUnit4importDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.CTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.GoogleTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
frompyEDAA.Reports.Unittesting.JUnit.PyTestJUnitimportDocument
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
Open Source VHDL Verification Methodology writes test results as YAML files for its
+internal data model storage. Some YAML files are written by the VHDL code of the verification framework, others are
+written by OSVVM-Scripts as a test runner.
A JUnit XML test report summary file can be read by creating an instance of the Document
+class. Because JUnit has so many dialects, a derived subclass for the dialect might be required. By choosing the
+right Document class, also the XML schema for XML schema validation gets pre-selected.
Any JUnit dialect specific data model can be converted to the generic hierarchy of test entities.
+
+
Note
+
This conversion is identical for all derived dialects.
+
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Read from XML file
+xmlReport=Path("JUnit-Report.xml")
+try:
+ doc=Document(xmlReport,parse=True)
+exceptUnittestExceptionasex:
+ ...
+
+# Convert to unified test data model
+summary=doc.ToTestsuiteSummary()
+
+# Convert to a tree
+rootNode=doc.ToTree()
+
+# Convert back to a document
+newXmlReport=Path("New JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+newDoc.Write()
+
A test suite summary can be converted to a document of any JUnit dialect. Internally a deep-copy is created to
+convert from a hierarchy of the unified test entities to a hierarchy of specific test entities (e.g. JUnit
+entities).
+
When the document was created, it can be written to disk.
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.AntJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.CTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.GoogleTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
+
+
+
+
+
frompyEDAA.Reports.Unittesting.JUnit.PyTestJUnitimportDocument
+
+# Convert a TestsuiteSummary back to a Document
+newXmlReport=Path("JUnit-Report.xml")
+newDoc=Document.FromTestsuiteSummary(newXmlReport,summary)
+
+# Write to XML file
+try:
+ newDoc.Write()
+exceptUnittestExceptionasex:
+ ...
+
Unittest summary reports can be stored in various file formats. Usually these files are XML based. Due to missing
+(clear) specifications and XML schema definitions, some file formats have developed dialects. Either because the
+specification was unclear/not existing or because the format was specific for a single programming language, so tools
+added extensions or misused XML attributes instead of designing their own file format.
The so-called JUnit XML format was defined by Ant when running JUnit4 test suites. Because the format was not specified
+by JUnit4, many dialects spread out. Many tools and test frameworks have minor or major differences compared to the
+original format. While some modifications seam logical additions or adjustments to the needs of the respective
+framework, others undermine the ideas and intents of the data format.
+
Many issues arise because the Ant + JUnit4 format is
+specific to unit testing with Java. Other languages and frameworks were lazy and didn’t derive their own format, but
+rather stuffed their language constructs into the concepts and limitations of the Ant + JUnit4 XML format.
JUnit5 uses a new format called Open Test Reporting (see the following section for details). This format
+isn’t specific to Java (packages, classes, methods, …), but describes a generic data model. Of cause an extension for
+Java specifics is provided too.
The Open Test Alliance created a new format called
+Open Test Reporting (OTR) to overcome the shortcommings of a
+missing file format for JUnit5 as well as the problems of Ant + JUnit4.
+
OTR defines a structure of test groups and tests, but no specifics of a certain programming languge. The logical
+structure of tests and test groups is decoupled from language specifics like namespaces, packages or classes hosting the
+individual tests.
The Open Source VHDL Verification Methodology (OSVVM) defines its own test report format
+in YAML. While OSVVM is able to convert its own YAML files to JUnit XML files, it’s recommended to use the YAML files as
+data source, because these contain additional information, which can’t be expressed with JUnit XML.
+
The YAML files are created when OSVVM-based testbenches are executed with OSVVM’s embedded TCL scripting environment
+OSVVM-Scripts.
+
+
Hint
+
YAML was chosen instead of JSON or XML, because a YAML document isn’t corrupted in case of a runtime error. The
+document might be incomplete (content), but not corrupted (structural). Such a scenario is possible if a VHDL
+simulator stops execution, then the document structure can’t be finalized.
Source code for pyEDAA.Reports.DocumentationCoverage.Python
+# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+**Abstract code documentation coverage data model for Python code.**
+"""
+frompathlibimportPath
+fromtypingimportOptionalasNullable,Iterable,Dict,Union,Tuple,List
+
+frompyTooling.Decoratorsimportexport,readonly
+frompyTooling.MetaClassesimportExtendedType
+
+frompyEDAA.Reports.DocumentationCoverageimportClass,Module,Package,CoverageState,DocCoverageException
+
+
+
+[docs]
+@export
+classCoverage(metaclass=ExtendedType,mixin=True):
+"""
+ This base-class for :class:`ClassCoverage` and :class:`AggregatedCoverage` represents a basic set of documentation coverage metrics.
+
+ Besides the *total* number of coverable items, it distinguishes items as *excluded*, *ignored*, and *expected*. |br|
+ Expected items are further distinguished into *covered* and *uncovered* items. |br|
+ If no item is expected, then *coverage* is always 100 |%|.
+
+ All coverable items
+ total = excluded + ignored + expected
+
+ All expected items
+ expected = covered + uncovered
+
+ Coverage [0.00..1.00]
+ coverage = covered / expected
+ """
+ _total:int
+ _excluded:int
+ _ignored:int
+ _expected:int
+ _covered:int
+ _uncovered:int
+
+ _coverage:float
+
+
+[docs]
+@export
+classAggregatedCoverage(Coverage,mixin=True):
+"""
+ This base-class for :class:`ModuleCoverage` and :class:`PackageCoverage` represents an extended set of documentation coverage metrics, especially with aggregated metrics.
+
+ As inherited from :class:`~Coverage`, it provides the *total* number of coverable items, which are distinguished into
+ *excluded*, *ignored*, and *expected* items. |br|
+ Expected items are further distinguished into *covered* and *uncovered* items. |br|
+ If no item is expected, then *coverage* and *aggregated coverage* are always 100 |%|.
+
+ In addition, all previously mentioned metrics are collected as *aggregated...*, too. |br|
+
+ All coverable items
+ total = excluded + ignored + expected
+
+ All expected items
+ expected = covered + uncovered
+
+ Coverage [0.00..1.00]
+ coverage = covered / expected
+ """
+ _file:Path
+
+ _aggregatedTotal:int
+ _aggregatedExcluded:int
+ _aggregatedIgnored:int
+ _aggregatedExpected:int
+ _aggregatedCovered:int
+ _aggregatedUncovered:int
+
+ _aggregatedCoverage:float
+
+
+[docs]
+@export
+classClassCoverage(Class,Coverage):
+"""
+ This class represents the class documentation coverage for Python classes.
+ """
+ _fields:Dict[str,CoverageState]
+ _methods:Dict[str,CoverageState]
+ _classes:Dict[str,"ClassCoverage"]
+
+
+# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+The pyEDAA.Reports.Unittesting package implements a hierarchy of test entities. These are test cases, test suites and a
+test summary provided as a class hierarchy. Test cases are the leaf elements in the hierarchy and abstract an
+individual test run. Test suites are used to group multiple test cases or other test suites. The root element is a test
+summary. When such a summary is stored in a file format like Ant + JUnit4 XML, a file format specific document is
+derived from a summary class.
+
+**Data Model**
+
+.. mermaid::
+
+ graph TD;
+ doc[Document]
+ sum[Summary]
+ ts1[Testsuite]
+ ts2[Testsuite]
+ ts21[Testsuite]
+ tc11[Testcase]
+ tc12[Testcase]
+ tc13[Testcase]
+ tc21[Testcase]
+ tc22[Testcase]
+ tc211[Testcase]
+ tc212[Testcase]
+ tc213[Testcase]
+
+ doc:::root -.-> sum:::summary
+ sum --> ts1:::suite
+ sum --> ts2:::suite
+ ts2 --> ts21:::suite
+ ts1 --> tc11:::case
+ ts1 --> tc12:::case
+ ts1 --> tc13:::case
+ ts2 --> tc21:::case
+ ts2 --> tc22:::case
+ ts21 --> tc211:::case
+ ts21 --> tc212:::case
+ ts21 --> tc213:::case
+
+ classDef root fill:#4dc3ff
+ classDef summary fill:#80d4ff
+ classDef suite fill:#b3e6ff
+ classDef case fill:#eeccff
+"""
+fromdatetimeimporttimedelta,datetime
+fromenumimportFlag,IntEnum
+frompathlibimportPath
+fromsysimportversion_info
+fromtypingimportOptionalasNullable,Dict,Iterable,Any,Tuple,Generator,Union,List,Generic,TypeVar,Mapping
+
+frompyTooling.CommonimportgetFullyQualifiedName
+frompyTooling.Decoratorsimportexport,readonly
+frompyTooling.MetaClassesimportExtendedType,abstractmethod
+frompyTooling.TreeimportNode
+
+frompyEDAA.ReportsimportReportException
+
+
+
+[docs]
+@export
+classUnittestException(ReportException):
+"""Base-exception for all unit test related exceptions."""
+
+
+
+
+[docs]
+@export
+classAlreadyInHierarchyException(UnittestException):
+"""
+ A unit test exception raised if the element is already part of a hierarchy.
+
+ This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+ hierarchy should occur only once in the hierarchy.
+
+ .. hint::
+
+ This is usually caused by a non-None parent reference.
+ """
+
+
+
+
+[docs]
+@export
+classDuplicateTestsuiteException(UnittestException):
+"""
+ A unit test exception raised on duplicate test suites (by name).
+
+ This exception is raised, if a child test suite with same name already exist in the test suite.
+
+ .. hint::
+
+ Test suite names need to be unique per parent element (test suite or test summary).
+ """
+
+
+
+
+[docs]
+@export
+classDuplicateTestcaseException(UnittestException):
+"""
+ A unit test exception raised on duplicate test cases (by name).
+
+ This exception is raised, if a child test case with same name already exist in the test suite.
+
+ .. hint::
+
+ Test case names need to be unique per parent element (test suite).
+ """
+
+
+
+
+[docs]
+@export
+classTestcaseStatus(Flag):
+"""A flag enumeration describing the status of a test case."""
+ Unknown=0#: Testcase status is uninitialized and therefore unknown.
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Weak=4#: No assertions were recorded.
+ Passed=8#: A passed testcase, because all assertions were successful.
+ Failed=16#: A failed testcase due to at least one failed assertion.
+
+ Mask=Excluded|Skipped|Weak|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+ Inconsistent=32768#: Dataset is inconsistent
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError|Inconsistent
+
+ # TODO: timed out ?
+
+ def__matmul__(self,other:"TestcaseStatus")->"TestcaseStatus":
+ s=self&self.Mask
+ o=other&self.Mask
+ ifsisself.Excluded:
+ resolved=self.Excludedifoisself.Excludedelseself.Unknown
+ elifsisself.Skipped:
+ resolved=self.Unknownif(oisself.Unknown)or(oisself.Excluded)elseo
+ elifsisself.Weak:
+ resolved=self.Weakifoisself.Weakelseself.Unknown
+ elifsisself.Passed:
+ resolved=self.Passedif(oisself.Skipped)or(oisself.Passed)elseself.Unknown
+ elifsisself.Failed:
+ resolved=self.Failedif(oisself.Skipped)or(oisself.Failed)elseself.Unknown
+ else:
+ resolved=self.Unknown
+
+ resolved|=(self&self.Flags)|(other&self.Flags)
+ returnresolved
+
+
+
+
+[docs]
+@export
+classTestsuiteStatus(Flag):
+"""A flag enumeration describing the status of a test suite."""
+ Unknown=0
+ Excluded=1#: Testcase was permanently excluded / disabled
+ Skipped=2#: Testcase was temporarily skipped (e.g. based on a condition)
+ Empty=4#: No tests in suite
+ Passed=8#: Passed testcase, because all assertions succeeded
+ Failed=16#: Failed testcase due to failing assertions
+
+ Mask=Excluded|Skipped|Empty|Passed|Failed
+
+ Inverted=128#: To mark inverted results
+ UnexpectedPassed=Failed|Inverted
+ ExpectedFailed=Passed|Inverted
+
+ Warned=1024#: Runtime warning
+ Errored=2048#: Runtime error (mostly caught exceptions)
+ Aborted=4096#: Uncaught runtime exception
+
+ SetupError=8192#: Preparation / compilation error
+ TearDownError=16384#: Cleanup error / resource release error
+
+ Flags=Warned|Errored|Aborted|SetupError|TearDownError
+
+
+
+
+[docs]
+@export
+classTestsuiteKind(IntEnum):
+"""Enumeration describing the kind of test suite."""
+ Root=0#: Root element of the hierarchy.
+ Logical=1#: Represents a logical unit.
+ Namespace=2#: Represents a namespace.
+ Package=3#: Represents a package.
+ Module=4#: Represents a module.
+ Class=5#: Represents a class.
+
+
+
+
+[docs]
+@export
+classIterationScheme(Flag):
+"""
+ A flag enumeration for selecting the test suite iteration scheme.
+
+ When a test entity hierarchy is (recursively) iterated, this iteration scheme describes how to iterate the hierarchy
+ and what elements to return as a result.
+ """
+ Unknown=0#: Neutral element.
+ IncludeSelf=1#: Also include the element itself.
+ IncludeTestsuites=2#: Include test suites into the result.
+ IncludeTestcases=4#: Include test cases into the result.
+
+ Recursive=8#: Iterate recursively.
+
+ PreOrder=16#: Iterate in pre-order (top-down: current node, then child element left-to-right).
+ PostOrder=32#: Iterate in pre-order (bottom-up: child element left-to-right, then current node).
+
+ Default=IncludeTestsuites|Recursive|IncludeTestcases|PreOrder#: Recursively iterate all test entities in pre-order.
+ TestsuiteDefault=IncludeTestsuites|Recursive|PreOrder#: Recursively iterate only test suites in pre-order.
+ TestcaseDefault=IncludeTestcases|Recursive|PreOrder#: Recursively iterate only test cases in pre-order.
+[docs]
+@export
+classBase(metaclass=ExtendedType,slots=True):
+"""
+ Base-class for all test entities (test cases, test suites, ...).
+
+ It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+ hierarchy.
+
+ Every test entity has a name to identity it. It's also used in the parent's child element dictionaries to identify the
+ child. |br|
+ E.g. it's used as a test case name in the dictionary of test cases in a test suite.
+
+ Every test entity has fields for time tracking. If known, a start time and a test duration can be set. For more
+ details, a setup duration and teardown duration can be added. All durations are summed up in a total duration field.
+
+ As tests can have warnings and errors or even fail, these messages are counted and aggregated in the test entity
+ hierarchy.
+
+ Every test entity offers an internal dictionary for annotations. |br|
+ This feature is for example used by Ant + JUnit4's XML property fields.
+ """
+
+ _parent:Nullable["TestsuiteBase"]
+ _name:str
+
+ _startTime:Nullable[datetime]
+ _setupDuration:Nullable[timedelta]
+ _testDuration:Nullable[timedelta]
+ _teardownDuration:Nullable[timedelta]
+ _totalDuration:Nullable[timedelta]
+
+ _warningCount:int
+ _errorCount:int
+ _fatalCount:int
+
+ _dict:Dict[str,Any]
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ startTime:Nullable[datetime]=None,
+ setupDuration:Nullable[timedelta]=None,
+ testDuration:Nullable[timedelta]=None,
+ teardownDuration:Nullable[timedelta]=None,
+ totalDuration:Nullable[timedelta]=None,
+ warningCount:int=0,
+ errorCount:int=0,
+ fatalCount:int=0,
+ keyValuePairs:Nullable[Mapping[str,Any]]=None,
+ parent:Nullable["TestsuiteBase"]=None
+ ):
+"""
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of the entity's test run.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown).
+ :param warningCount: Count of encountered warnings.
+ :param errorCount: Count of encountered errors.
+ :param fatalCount: Count of encountered fatal errors.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test entity with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ :raises ValueError: If parameter 'name' is None.
+ :raises TypeError: If parameter 'name' is not a string.
+ :raises ValueError: If parameter 'name' is empty.
+ :raises TypeError: If parameter 'testDuration' is not a timedelta.
+ :raises TypeError: If parameter 'setupDuration' is not a timedelta.
+ :raises TypeError: If parameter 'teardownDuration' is not a timedelta.
+ :raises TypeError: If parameter 'totalDuration' is not a timedelta.
+ :raises TypeError: If parameter 'warningCount' is not an integer.
+ :raises TypeError: If parameter 'errorCount' is not an integer.
+ :raises TypeError: If parameter 'fatalCount' is not an integer.
+ :raises TypeError: If parameter 'keyValuePairs' is not a Mapping.
+ :raises ValueError: If parameter 'totalDuration' is not consistent.
+ """
+
+ ifparentisnotNoneandnotisinstance(parent,TestsuiteBase):
+ ex=TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ ifnameisNone:
+ raiseValueError(f"Parameter 'name' is None.")
+ elifnotisinstance(name,str):
+ ex=TypeError(f"Parameter 'name' is not of type 'str'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(name)}'.")
+ raiseex
+ elifname.strip()=="":
+ raiseValueError(f"Parameter 'name' is empty.")
+
+ self._parent=parent
+ self._name=name
+
+ iftestDurationisnotNoneandnotisinstance(testDuration,timedelta):
+ ex=TypeError(f"Parameter 'testDuration' is not of type 'timedelta'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testDuration)}'.")
+ raiseex
+
+ ifsetupDurationisnotNoneandnotisinstance(setupDuration,timedelta):
+ ex=TypeError(f"Parameter 'setupDuration' is not of type 'timedelta'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(setupDuration)}'.")
+ raiseex
+
+ ifteardownDurationisnotNoneandnotisinstance(teardownDuration,timedelta):
+ ex=TypeError(f"Parameter 'teardownDuration' is not of type 'timedelta'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(teardownDuration)}'.")
+ raiseex
+
+ iftotalDurationisnotNoneandnotisinstance(totalDuration,timedelta):
+ ex=TypeError(f"Parameter 'totalDuration' is not of type 'timedelta'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(totalDuration)}'.")
+ raiseex
+
+ iftestDurationisnotNone:
+ ifsetupDurationisnotNone:
+ ifteardownDurationisnotNone:
+ iftotalDurationisnotNone:
+ iftotalDuration<(setupDuration+testDuration+teardownDuration):
+ raiseValueError(f"Parameter 'totalDuration' can not be less than the sum of setup, test and teardown durations.")
+ else:# no total
+ totalDuration=setupDuration+testDuration+teardownDuration
+ # no teardown
+ eliftotalDurationisnotNone:
+ iftotalDuration<(setupDuration+testDuration):
+ raiseValueError(f"Parameter 'totalDuration' can not be less than the sum of setup and test durations.")
+ # no teardown, no total
+ else:
+ totalDuration=setupDuration+testDuration
+ # no setup
+ elifteardownDurationisnotNone:
+ iftotalDurationisnotNone:
+ iftotalDuration<(testDuration+teardownDuration):
+ raiseValueError(f"Parameter 'totalDuration' can not be less than the sum of test and teardown durations.")
+ else:# no setup, no total
+ totalDuration=testDuration+teardownDuration
+ # no setup, no teardown
+ eliftotalDurationisnotNone:
+ iftotalDuration<testDuration:
+ raiseValueError(f"Parameter 'totalDuration' can not be less than test durations.")
+ else:# no setup, no teardown, no total
+ totalDuration=testDuration
+ # no test
+ eliftotalDurationisnotNone:
+ testDuration=totalDuration
+ ifsetupDurationisnotNone:
+ testDuration-=setupDuration
+ ifteardownDurationisnotNone:
+ testDuration-=teardownDuration
+
+ self._startTime=startTime
+ self._setupDuration=setupDuration
+ self._testDuration=testDuration
+ self._teardownDuration=teardownDuration
+ self._totalDuration=totalDuration
+
+ ifnotisinstance(warningCount,int):
+ ex=TypeError(f"Parameter 'warningCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(warningCount)}'.")
+ raiseex
+
+ ifnotisinstance(errorCount,int):
+ ex=TypeError(f"Parameter 'errorCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(errorCount)}'.")
+ raiseex
+
+ ifnotisinstance(fatalCount,int):
+ ex=TypeError(f"Parameter 'fatalCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(fatalCount)}'.")
+ raiseex
+
+ self._warningCount=warningCount
+ self._errorCount=errorCount
+ self._fatalCount=fatalCount
+
+ ifkeyValuePairsisnotNoneandnotisinstance(keyValuePairs,Mapping):
+ ex=TypeError(f"Parameter 'keyValuePairs' is not a mapping.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(keyValuePairs)}'.")
+ raiseex
+
+ self._dict={}ifkeyValuePairsisNoneelse{k:vfork,vinkeyValuePairs}
+
+
+ # QUESTION: allow Parent as setter?
+ @readonly
+ defParent(self)->Nullable["TestsuiteBase"]:
+"""
+ Read-only property returning the reference to the parent test entity.
+
+ :return: Reference to the parent entity.
+ """
+ returnself._parent
+
+ @readonly
+ defName(self)->str:
+"""
+ Read-only property returning the test entity's name.
+
+ :return:
+ """
+ returnself._name
+
+ @readonly
+ defStartTime(self)->Nullable[datetime]:
+"""
+ Read-only property returning the time when the test entity was started.
+
+ :return: Time when the test entity was started.
+ """
+ returnself._startTime
+
+ @readonly
+ defSetupDuration(self)->Nullable[timedelta]:
+"""
+ Read-only property returning the duration of the test entity's setup.
+
+ :return: Duration it took to set up the entity.
+ """
+ returnself._setupDuration
+
+ @readonly
+ defTestDuration(self)->Nullable[timedelta]:
+"""
+ Read-only property returning the duration of a test entities run.
+
+ This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+ distinguishable, assign setup and teardown durations with zero.
+
+ :return: Duration of the entity's test run.
+ """
+ returnself._testDuration
+
+ @readonly
+ defTeardownDuration(self)->Nullable[timedelta]:
+"""
+ Read-only property returning the duration of the test entity's teardown.
+
+ :return: Duration it took to tear down the entity.
+ """
+ returnself._teardownDuration
+
+ @readonly
+ defTotalDuration(self)->Nullable[timedelta]:
+"""
+ Read-only property returning the total duration of a test entity run.
+
+ this duration includes setup and teardown durations.
+
+ :return: Total duration of the entity's execution (setup + test + teardown)
+ """
+ returnself._totalDuration
+
+ @readonly
+ defWarningCount(self)->int:
+"""
+ Read-only property returning the number of encountered warnings.
+
+ :return: Count of encountered warnings.
+ """
+ returnself._warningCount
+
+ @readonly
+ defErrorCount(self)->int:
+"""
+ Read-only property returning the number of encountered errors.
+
+ :return: Count of encountered errors.
+ """
+ returnself._errorCount
+
+ @readonly
+ defFatalCount(self)->int:
+"""
+ Read-only property returning the number of encountered fatal errors.
+
+ :return: Count of encountered fatal errors.
+ """
+ returnself._fatalCount
+
+
+[docs]
+ def__len__(self)->int:
+"""
+ Returns the number of annotated key-value pairs.
+
+ :return: Number of annotated key-value pairs.
+ """
+ returnlen(self._dict)
+
+
+
+[docs]
+ def__getitem__(self,key:str)->Any:
+"""
+ Access a key-value pair by key.
+
+ :param key: Name if the key-value pair.
+ :return: Value of the accessed key.
+ """
+ returnself._dict[key]
+
+
+
+[docs]
+ def__setitem__(self,key:str,value:Any)->None:
+"""
+ Set the value of a key-value pair by key.
+
+ If the pair doesn't exist yet, it's created.
+
+ :param key: Key of the key-value pair.
+ :param value: Value of the key-value pair.
+ """
+ self._dict[key]=value
+
+
+
+[docs]
+ def__delitem__(self,key:str)->None:
+"""
+ Delete a key-value pair by key.
+
+ :param key: Name if the key-value pair.
+ """
+ delself._dict[key]
+
+
+
+[docs]
+ def__contains__(self,key:str)->bool:
+"""
+ Returns True, if a key-value pairs was annotated by this key.
+
+ :param key: Name of the key-value pair.
+ :return: True, if the pair was annotated.
+ """
+ returnkeyinself._dict
+
+
+
+[docs]
+ def__iter__(self)->Generator[Tuple[str,Any],None,None]:
+"""
+ Iterate all annotated key-value pairs.
+
+ :return: A generator of key-value pair tuples (key, value).
+ """
+ yield fromself._dict.items()
+
+
+
+[docs]
+ @abstractmethod
+ defAggregate(self,strict:bool=True):
+"""
+ Aggregate all test entities in the hierarchy.
+
+ :return:
+ """
+
+
+
+[docs]
+ @abstractmethod
+ def__str__(self)->str:
+"""
+ Formats the test entity as human-readable incl. some statistics.
+ """
+
+
+
+
+
+[docs]
+@export
+classTestcase(Base):
+"""
+ A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
+ Test cases are grouped by test suites in the test entity hierarchy. The root of the hierarchy is a test summary.
+
+ Every test case has an overall status like unknown, skipped, failed or passed.
+
+ In addition to all features from its base-class, test cases provide additional statistics for passed and failed
+ assertions (checks) as well as a sum thereof.
+ """
+
+ _status:TestcaseStatus
+ _assertionCount:Nullable[int]
+ _failedAssertionCount:Nullable[int]
+ _passedAssertionCount:Nullable[int]
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ startTime:Nullable[datetime]=None,
+ setupDuration:Nullable[timedelta]=None,
+ testDuration:Nullable[timedelta]=None,
+ teardownDuration:Nullable[timedelta]=None,
+ totalDuration:Nullable[timedelta]=None,
+ status:TestcaseStatus=TestcaseStatus.Unknown,
+ assertionCount:Nullable[int]=None,
+ failedAssertionCount:Nullable[int]=None,
+ passedAssertionCount:Nullable[int]=None,
+ warningCount:int=0,
+ errorCount:int=0,
+ fatalCount:int=0,
+ keyValuePairs:Nullable[Mapping[str,Any]]=None,
+ parent:Nullable["Testsuite"]=None
+ ):
+"""
+ Initializes the fields of a test case.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of the entity's test run.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Status of the test case.
+ :param assertionCount: Number of assertions within the test.
+ :param failedAssertionCount: Number of failed assertions within the test.
+ :param passedAssertionCount: Number of passed assertions within the test.
+ :param warningCount: Count of encountered warnings.
+ :param errorCount: Count of encountered errors.
+ :param fatalCount: Count of encountered fatal errors.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test case.
+ :param parent: Reference to the parent test suite.
+ :raises TypeError: If parameter 'parent' is not a Testsuite.
+ :raises ValueError: If parameter 'assertionCount' is not consistent.
+ """
+
+ ifparentisnotNone:
+ ifnotisinstance(parent,Testsuite):
+ ex=TypeError(f"Parameter 'parent' is not of type 'Testsuite'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testcases[name]=self
+
+ super().__init__(
+ name,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ warningCount,
+ errorCount,
+ fatalCount,
+ keyValuePairs,
+ parent
+ )
+
+ ifnotisinstance(status,TestcaseStatus):
+ ex=TypeError(f"Parameter 'status' is not of type 'TestcaseStatus'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(status)}'.")
+ raiseex
+
+ self._status=status
+
+ ifassertionCountisnotNoneandnotisinstance(assertionCount,int):
+ ex=TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(assertionCount)}'.")
+ raiseex
+
+ iffailedAssertionCountisnotNoneandnotisinstance(failedAssertionCount,int):
+ ex=TypeError(f"Parameter 'failedAssertionCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(failedAssertionCount)}'.")
+ raiseex
+
+ ifpassedAssertionCountisnotNoneandnotisinstance(passedAssertionCount,int):
+ ex=TypeError(f"Parameter 'passedAssertionCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(passedAssertionCount)}'.")
+ raiseex
+
+ self._assertionCount=assertionCount
+ ifassertionCountisnotNone:
+ iffailedAssertionCountisnotNone:
+ self._failedAssertionCount=failedAssertionCount
+
+ ifpassedAssertionCountisnotNone:
+ ifpassedAssertionCount+failedAssertionCount!=assertionCount:
+ raiseValueError(f"passed assertion count ({passedAssertionCount}) + failed assertion count ({failedAssertionCount} != assertion count ({assertionCount}")
+
+ self._passedAssertionCount=passedAssertionCount
+ else:
+ self._passedAssertionCount=assertionCount-failedAssertionCount
+ elifpassedAssertionCountisnotNone:
+ self._passedAssertionCount=passedAssertionCount
+ self._failedAssertionCount=assertionCount-passedAssertionCount
+ else:
+ raiseValueError(f"Neither passed assertion count nor failed assertion count are provided.")
+ eliffailedAssertionCountisnotNone:
+ self._failedAssertionCount=failedAssertionCount
+
+ ifpassedAssertionCountisnotNone:
+ self._passedAssertionCount=passedAssertionCount
+ self._assertionCount=passedAssertionCount+failedAssertionCount
+ else:
+ raiseValueError(f"Passed assertion count is mandatory, if failed assertion count is provided instead of assertion count.")
+ elifpassedAssertionCountisnotNone:
+ raiseValueError(f"Assertion count or failed assertion count is mandatory, if passed assertion count is provided.")
+ else:
+ self._passedAssertionCount=None
+ self._failedAssertionCount=None
+
+
+ @readonly
+ defStatus(self)->TestcaseStatus:
+"""
+ Read-only property returning the status of the test case.
+
+ :return: The test case's status.
+ """
+ returnself._status
+
+ @readonly
+ defAssertionCount(self)->int:
+"""
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ :return: Number of assertions.
+ """
+ ifself._assertionCountisNone:
+ return0
+ returnself._assertionCount
+
+ @readonly
+ defFailedAssertionCount(self)->int:
+"""
+ Read-only property returning the number of failed assertions (failed checks) in a test case.
+
+ :return: Number of assertions.
+ """
+ returnself._failedAssertionCount
+
+ @readonly
+ defPassedAssertionCount(self)->int:
+"""
+ Read-only property returning the number of passed assertions (successful checks) in a test case.
+
+ :return: Number of passed assertions.
+ """
+ returnself._passedAssertionCount
+
+ defCopy(self)->"Testcase":
+ returnself.__class__(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._testDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount,
+ )
+
+
+[docs]
+ def__str__(self)->str:
+"""
+ Formats the test case as human-readable incl. statistics.
+
+ :pycode:`f"<Testcase {}: {} - assert/pass/fail:{}/{}/{} - warn/error/fatal:{}/{}/{} - setup/test/teardown:{}/{}/{}>"`
+
+ :return: Human-readable summary of a test case object.
+ """
+ return(
+ f"<Testcase {self._name}: {self._status.name} -"
+ f" assert/pass/fail:{self._assertionCount}/{self._passedAssertionCount}/{self._failedAssertionCount} -"
+ f" warn/error/fatal:{self._warningCount}/{self._errorCount}/{self._fatalCount} -"
+ f" setup/test/teardown:{self._setupDuration:.3f}/{self._testDuration:.3f}/{self._teardownDuration:.3f}>"
+ )
+
+
+
+
+
+[docs]
+@export
+classTestsuiteBase(Base,Generic[TestsuiteType]):
+"""
+ Base-class for all test suites and for test summaries.
+
+ A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+ element in that hierarchy. While a test suite groups other test suites and test cases, a test summary can only group
+ test suites. Thus, a test summary contains no test cases.
+ """
+
+ _kind:TestsuiteKind
+ _status:TestsuiteStatus
+ _testsuites:Dict[str,TestsuiteType]
+
+ _tests:int
+ _inconsistent:int
+ _excluded:int
+ _skipped:int
+ _errored:int
+ _weak:int
+ _failed:int
+ _passed:int
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ kind:TestsuiteKind=TestsuiteKind.Logical,
+ startTime:Nullable[datetime]=None,
+ setupDuration:Nullable[timedelta]=None,
+ testDuration:Nullable[timedelta]=None,
+ teardownDuration:Nullable[timedelta]=None,
+ totalDuration:Nullable[timedelta]=None,
+ status:TestsuiteStatus=TestsuiteStatus.Unknown,
+ warningCount:int=0,
+ errorCount:int=0,
+ fatalCount:int=0,
+ testsuites:Nullable[Iterable[TestsuiteType]]=None,
+ keyValuePairs:Nullable[Mapping[str,Any]]=None,
+ parent:Nullable["Testsuite"]=None
+ ):
+"""
+ Initializes the based-class fields of a test suite or test summary.
+
+ :param name: Name of the test entity.
+ :param kind: Kind of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of all tests listed in the test entity.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test entity.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test entity with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test entity with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ :raises TypeError: If parameter 'testsuites' is not iterable.
+ :raises TypeError: If element in parameter 'testsuites' is not a Testsuite.
+ :raises AlreadyInHierarchyException: If a test suite in parameter 'testsuites' is already part of a test entity hierarchy.
+ :raises DuplicateTestsuiteException: If a test suite in parameter 'testsuites' is already listed (by name) in the list of test suites.
+ """
+ ifparentisnotNone:
+ ifnotisinstance(parent,TestsuiteBase):
+ ex=TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testsuites[name]=self
+
+ super().__init__(
+ name,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ warningCount,
+ errorCount,
+ fatalCount,
+ keyValuePairs,
+ parent
+ )
+
+ self._kind=kind
+ self._status=status
+
+ self._testsuites={}
+ iftestsuitesisnotNone:
+ ifnotisinstance(testsuites,Iterable):
+ ex=TypeError(f"Parameter 'testsuites' is not iterable.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuites)}'.")
+ raiseex
+
+ fortestsuiteintestsuites:
+ ifnotisinstance(testsuite,Testsuite):
+ ex=TypeError(f"Element of parameter 'testsuites' is not of type 'Testsuite'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuite)}'.")
+ raiseex
+
+ iftestsuite._parentisnotNone:
+ raiseAlreadyInHierarchyException(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ iftestsuite._nameinself._testsuites:
+ raiseDuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent=self
+ self._testsuites[testsuite._name]=testsuite
+
+ self._status=TestsuiteStatus.Unknown
+ self._tests=0
+ self._inconsistent=0
+ self._excluded=0
+ self._skipped=0
+ self._errored=0
+ self._weak=0
+ self._failed=0
+ self._passed=0
+
+
+ @readonly
+ defKind(self)->TestsuiteKind:
+"""
+ Read-only property returning the kind of the test suite.
+
+ Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+ grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+ language construct.
+
+ Test summaries always return kind ``Root``.
+
+ :return: Kind of the test suite.
+ """
+ returnself._kind
+
+ @readonly
+ defStatus(self)->TestsuiteStatus:
+"""
+ Read-only property returning the aggregated overall status of the test suite.
+
+ :return: Overall status of the test suite.
+ """
+ returnself._status
+
+ @readonly
+ defTestsuites(self)->Dict[str,TestsuiteType]:
+"""
+ Read-only property returning a reference to the internal dictionary of test suites.
+
+ :return: Reference to the dictionary of test suite.
+ """
+ returnself._testsuites
+
+ @readonly
+ defTestsuiteCount(self)->int:
+"""
+ Read-only property returning the number of all test suites in the test suite hierarchy.
+
+ :return: Number of test suites.
+ """
+ return1+sum(testsuite.TestsuiteCountfortestsuiteinself._testsuites.values())
+
+ @readonly
+ defTestcaseCount(self)->int:
+"""
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ returnsum(testsuite.TestcaseCountfortestsuiteinself._testsuites.values())
+
+ @readonly
+ defAssertionCount(self)->int:
+"""
+ Read-only property returning the number of all assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of assertions in all test cases.
+ """
+ returnsum(ts.AssertionCountfortsinself._testsuites.values())
+
+ @readonly
+ defFailedAssertionCount(self)->int:
+"""
+ Read-only property returning the number of all failed assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of failed assertions in all test cases.
+ """
+ raiseNotImplementedError()
+ # return self._assertionCount - (self._warningCount + self._errorCount + self._fatalCount)
+
+ @readonly
+ defPassedAssertionCount(self)->int:
+"""
+ Read-only property returning the number of all passed assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of passed assertions in all test cases.
+ """
+ raiseNotImplementedError()
+ # return self._assertionCount - (self._warningCount + self._errorCount + self._fatalCount)
+
+ @readonly
+ defTests(self)->int:
+ returnself._tests
+
+ @readonly
+ defInconsistent(self)->int:
+"""
+ Read-only property returning the number of inconsistent tests in the test suite hierarchy.
+
+ :return: Number of inconsistent tests.
+ """
+ returnself._inconsistent
+
+ @readonly
+ defExcluded(self)->int:
+"""
+ Read-only property returning the number of excluded tests in the test suite hierarchy.
+
+ :return: Number of excluded tests.
+ """
+ returnself._excluded
+
+ @readonly
+ defSkipped(self)->int:
+"""
+ Read-only property returning the number of skipped tests in the test suite hierarchy.
+
+ :return: Number of skipped tests.
+ """
+ returnself._skipped
+
+ @readonly
+ defErrored(self)->int:
+"""
+ Read-only property returning the number of tests with errors in the test suite hierarchy.
+
+ :return: Number of errored tests.
+ """
+ returnself._errored
+
+ @readonly
+ defWeak(self)->int:
+"""
+ Read-only property returning the number of weak tests in the test suite hierarchy.
+
+ :return: Number of weak tests.
+ """
+ returnself._weak
+
+ @readonly
+ defFailed(self)->int:
+"""
+ Read-only property returning the number of failed tests in the test suite hierarchy.
+
+ :return: Number of failed tests.
+ """
+ returnself._failed
+
+ @readonly
+ defPassed(self)->int:
+"""
+ Read-only property returning the number of passed tests in the test suite hierarchy.
+
+ :return: Number of passed tests.
+ """
+ returnself._passed
+
+ @readonly
+ defWarningCount(self)->int:
+ raiseNotImplementedError()
+ # return self._warningCount
+
+ @readonly
+ defErrorCount(self)->int:
+ raiseNotImplementedError()
+ # return self._errorCount
+
+ @readonly
+ defFatalCount(self)->int:
+ raiseNotImplementedError()
+ # return self._fatalCount
+
+
+[docs]
+ defAddTestsuite(self,testsuite:TestsuiteType)->None:
+"""
+ Add a test suite to the list of test suites.
+
+ :param testsuite: The test suite to add.
+ :raises ValueError: If parameter 'testsuite' is None.
+ :raises TypeError: If parameter 'testsuite' is not a Testsuite.
+ :raises AlreadyInHierarchyException: If parameter 'testsuite' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If parameter 'testsuite' is already listed (by name) in the list of test suites.
+ """
+ iftestsuiteisNone:
+ raiseValueError("Parameter 'testsuite' is None.")
+ elifnotisinstance(testsuite,Testsuite):
+ ex=TypeError(f"Parameter 'testsuite' is not of type 'Testsuite'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuite)}'.")
+ raiseex
+
+ iftestsuite._parentisnotNone:
+ raiseAlreadyInHierarchyException(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ iftestsuite._nameinself._testsuites:
+ raiseDuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent=self
+ self._testsuites[testsuite._name]=testsuite
+
+
+
+[docs]
+ defAddTestsuites(self,testsuites:Iterable[TestsuiteType])->None:
+"""
+ Add a list of test suites to the list of test suites.
+
+ :param testsuites: List of test suites to add.
+ :raises ValueError: If parameter 'testsuites' is None.
+ :raises TypeError: If parameter 'testsuites' is not iterable.
+ """
+ iftestsuitesisNone:
+ raiseValueError("Parameter 'testsuites' is None.")
+ elifnotisinstance(testsuites,Iterable):
+ ex=TypeError(f"Parameter 'testsuites' is not iterable.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuites)}'.")
+ raiseex
+
+ fortestsuiteintestsuites:
+ self.AddTestsuite(testsuite)
+[docs]
+@export
+classTestsuite(TestsuiteBase[TestsuiteType]):
+"""
+ A testsuite is a mid-level element in the test entity hierarchy representing a group of tests.
+
+ Test suites contain test cases and optionally other test suites. Test suites can be grouped by test suites to form a
+ hierarchy of test entities. The root of the hierarchy is a test summary.
+ """
+
+ _testcases:Dict[str,"Testcase"]
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ kind:TestsuiteKind=TestsuiteKind.Logical,
+ startTime:Nullable[datetime]=None,
+ setupDuration:Nullable[timedelta]=None,
+ testDuration:Nullable[timedelta]=None,
+ teardownDuration:Nullable[timedelta]=None,
+ totalDuration:Nullable[timedelta]=None,
+ status:TestsuiteStatus=TestsuiteStatus.Unknown,
+ warningCount:int=0,
+ errorCount:int=0,
+ fatalCount:int=0,
+ testsuites:Nullable[Iterable[TestsuiteType]]=None,
+ testcases:Nullable[Iterable["Testcase"]]=None,
+ keyValuePairs:Nullable[Mapping[str,Any]]=None,
+ parent:Nullable[TestsuiteType]=None
+ ):
+"""
+ Initializes the fields of a test suite.
+
+ :param name: Name of the test suite.
+ :param kind: Kind of the test suite.
+ :param startTime: Time when the test suite was started.
+ :param setupDuration: Duration it took to set up the test suite.
+ :param testDuration: Duration of all tests listed in the test suite.
+ :param teardownDuration: Duration it took to tear down the test suite.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test suite.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test suite with.
+ :param testcases: List of test cases to initialize the test suite with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test suite with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ :raises TypeError: If element in parameter 'testcases' is not a Testcase.
+ :raises AlreadyInHierarchyException: If a test case in parameter 'testcases' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If a test case in parameter 'testcases' is already listed (by name) in the list of test cases.
+ """
+ super().__init__(
+ name,
+ kind,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ status,
+ warningCount,
+ errorCount,
+ fatalCount,
+ testsuites,
+ keyValuePairs,
+ parent
+ )
+
+ # self._testDuration = testDuration
+
+ self._testcases={}
+ iftestcasesisnotNone:
+ ifnotisinstance(testcases,Iterable):
+ ex=TypeError(f"Parameter 'testcases' is not iterable.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcases)}'.")
+ raiseex
+
+ fortestcaseintestcases:
+ ifnotisinstance(testcase,Testcase):
+ ex=TypeError(f"Element of parameter 'testcases' is not of type 'Testcase'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcase)}'.")
+ raiseex
+
+ iftestcase._parentisnotNone:
+ raiseAlreadyInHierarchyException(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ iftestcase._nameinself._testcases:
+ raiseDuplicateTestcaseException(f"Testsuite already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent=self
+ self._testcases[testcase._name]=testcase
+
+
+ @readonly
+ defTestcases(self)->Dict[str,"Testcase"]:
+"""
+ Read-only property returning a reference to the internal dictionary of test cases.
+
+ :return: Reference to the dictionary of test cases.
+ """
+ returnself._testcases
+
+ @readonly
+ defTestcaseCount(self)->int:
+"""
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ returnsuper().TestcaseCount+len(self._testcases)
+
+ @readonly
+ defAssertionCount(self)->int:
+ returnsuper().AssertionCount+sum(tc.AssertionCountfortcinself._testcases.values())
+
+ defCopy(self)->"Testsuite":
+ returnself.__class__(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount
+ )
+
+
+[docs]
+ defAddTestcase(self,testcase:"Testcase")->None:
+"""
+ Add a test case to the list of test cases.
+
+ :param testcase: The test case to add.
+ :raises ValueError: If parameter 'testcase' is None.
+ :raises TypeError: If parameter 'testcase' is not a Testcase.
+ :raises AlreadyInHierarchyException: If parameter 'testcase' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If parameter 'testcase' is already listed (by name) in the list of test cases.
+ """
+ iftestcaseisNone:
+ raiseValueError("Parameter 'testcase' is None.")
+ elifnotisinstance(testcase,Testcase):
+ ex=TypeError(f"Parameter 'testcase' is not of type 'Testcase'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcase)}'.")
+ raiseex
+
+ iftestcase._parentisnotNone:
+ raiseValueError(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ iftestcase._nameinself._testcases:
+ raiseDuplicateTestcaseException(f"Testsuite already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent=self
+ self._testcases[testcase._name]=testcase
+
+
+
+[docs]
+ defAddTestcases(self,testcases:Iterable["Testcase"])->None:
+"""
+ Add a list of test cases to the list of test cases.
+
+ :param testcases: List of test cases to add.
+ :raises ValueError: If parameter 'testcases' is None.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ """
+ iftestcasesisNone:
+ raiseValueError("Parameter 'testcases' is None.")
+ elifnotisinstance(testcases,Iterable):
+ ex=TypeError(f"Parameter 'testcases' is not iterable.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcases)}'.")
+ raiseex
+
+ fortestcaseintestcases:
+ self.AddTestcase(testcase)
+[docs]
+@export
+classTestsuiteSummary(TestsuiteBase[TestsuiteType]):
+"""
+ A testsuite summary is the root element in the test entity hierarchy representing a summary of all test suites and cases.
+
+ The testsuite summary contains test suites, which in turn can contain test suites and test cases.
+ """
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ startTime:Nullable[datetime]=None,
+ setupDuration:Nullable[timedelta]=None,
+ testDuration:Nullable[timedelta]=None,
+ teardownDuration:Nullable[timedelta]=None,
+ totalDuration:Nullable[timedelta]=None,
+ status:TestsuiteStatus=TestsuiteStatus.Unknown,
+ warningCount:int=0,
+ errorCount:int=0,
+ fatalCount:int=0,
+ testsuites:Nullable[Iterable[TestsuiteType]]=None,
+ keyValuePairs:Nullable[Mapping[str,Any]]=None,
+ parent:Nullable[TestsuiteType]=None
+ ):
+"""
+ Initializes the fields of a test summary.
+
+ :param name: Name of the test summary.
+ :param startTime: Time when the test summary was started.
+ :param setupDuration: Duration it took to set up the test summary.
+ :param testDuration: Duration of all tests listed in the test summary.
+ :param teardownDuration: Duration it took to tear down the test summary.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test summary.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test summary with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test summary with.
+ :param parent: Reference to the parent test summary.
+ """
+ super().__init__(
+ name,
+ TestsuiteKind.Root,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ status,
+ warningCount,
+ errorCount,
+ fatalCount,
+ testsuites,
+ keyValuePairs,
+ parent
+ )
+[docs]
+@export
+classDocument(metaclass=ExtendedType,mixin=True):
+"""A mixin-class representing a unit test summary document (file)."""
+
+ _path:Path
+
+ _analysisDuration:float#: TODO: replace by Timer; should be timedelta?
+ _modelConversion:float#: TODO: replace by Timer; should be timedelta?
+
+
+
+
+ @readonly
+ defPath(self)->Path:
+"""
+ Read-only property returning the path to the file of this document.
+
+ :return: The document's path to the file.
+ """
+ returnself._path
+
+ @readonly
+ defAnalysisDuration(self)->timedelta:
+"""
+ Read-only property returning analysis duration.
+
+ .. note::
+
+ This includes usually the duration to validate and parse the file format, but it excludes the time to convert the
+ content to the test entity hierarchy.
+
+ :return: Duration to analyze the document.
+ """
+ returntimedelta(seconds=self._analysisDuration)
+
+ @readonly
+ defModelConversionDuration(self)->timedelta:
+"""
+ Read-only property returning conversion duration.
+
+ .. note::
+
+ This includes usually the duration to convert the document's content to the test entity hierarchy. It might also
+ include the duration to (re-)aggregate all states and statistics in the hierarchy.
+
+ :return: Duration to convert the document.
+ """
+ returntimedelta(seconds=self._modelConversion)
+
+
+[docs]
+ @abstractmethod
+ defAnalyze(self)->None:
+"""Analyze and validate the document's content."""
+# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+The pyEDAA.Reports.Unittesting.JUnit package implements a hierarchy of test entities for the JUnit unit testing summary
+file format (XML format). This test entity hierarchy is not derived from :class:`pyEDAA.Reports.Unittesting`, because it
+doesn't match the unified data model. Nonetheless, both data models can be converted to each other. In addition, derived
+data models are provided for the many dialects of that XML file format. See the list modules in this package for the
+implemented dialects.
+
+The test entity hierarchy consists of test cases, test classes, test suites and a test summary. Test cases are the leaf
+elements in the hierarchy and represent an individual test run. Next, test classes group test cases, because the
+original Ant + JUnit format groups test cases (Java methods) in a Java class. Next, test suites are used to group
+multiple test classes. Finally, the root element is a test summary. When such a summary is stored in a file format like
+Ant + JUnit4 XML, a file format specific document is derived from a summary class.
+
+**Data Model**
+
+.. mermaid::
+
+ graph TD;
+ doc[Document]
+ sum[Summary]
+ ts1[Testsuite]
+ ts11[Testsuite]
+ ts2[Testsuite]
+
+ tc111[Testclass]
+ tc112[Testclass]
+ tc23[Testclass]
+
+ tc1111[Testcase]
+ tc1112[Testcase]
+ tc1113[Testcase]
+ tc1121[Testcase]
+ tc1122[Testcase]
+ tc231[Testcase]
+ tc232[Testcase]
+ tc233[Testcase]
+
+ doc:::root -.-> sum:::summary
+ sum --> ts1:::suite
+ sum ---> ts2:::suite
+ ts1 --> ts11:::suite
+
+ ts11 --> tc111:::cls
+ ts11 --> tc112:::cls
+ ts2 --> tc23:::cls
+
+ tc111 --> tc1111:::case
+ tc111 --> tc1112:::case
+ tc111 --> tc1113:::case
+ tc112 --> tc1121:::case
+ tc112 --> tc1122:::case
+ tc23 --> tc231:::case
+ tc23 --> tc232:::case
+ tc23 --> tc233:::case
+
+ classDef root fill:#4dc3ff
+ classDef summary fill:#80d4ff
+ classDef suite fill:#b3e6ff
+ classDef cls fill:#ff9966
+ classDef case fill:#eeccff
+"""
+fromdatetimeimportdatetime,timedelta
+fromenumimportFlag
+frompathlibimportPath
+fromsysimportversion_info
+fromtimeimportperf_counter_ns
+fromtypingimportOptionalasNullable,Iterable,Dict,Any,Generator,Tuple,Union,TypeVar,Type,ClassVar
+
+fromlxml.etreeimportXMLParser,parse,XMLSchema,ElementTree,Element,SubElement,tostring
+fromlxml.etreeimportXMLSyntaxError,_ElementTree,_Element,_Comment,XMLSchemaParseError
+frompyTooling.CommonimportgetFullyQualifiedName,getResourceFile
+frompyTooling.Decoratorsimportexport,readonly
+frompyTooling.ExceptionsimportToolingException
+frompyTooling.MetaClassesimportExtendedType,mustoverride,abstractmethod
+frompyTooling.TreeimportNode
+
+frompyEDAA.ReportsimportResources
+frompyEDAA.Reports.UnittestingimportUnittestException,AlreadyInHierarchyException,DuplicateTestsuiteException,DuplicateTestcaseException
+frompyEDAA.Reports.UnittestingimportTestcaseStatus,TestsuiteStatus,TestsuiteKind,IterationScheme
+frompyEDAA.Reports.UnittestingimportDocumentasut_Document,TestsuiteSummaryasut_TestsuiteSummary
+frompyEDAA.Reports.UnittestingimportTestsuiteasut_Testsuite,Testcaseasut_Testcase
+
+
+
+[docs]
+@export
+classJUnitException:
+"""An exception-mixin for JUnit format specific exceptions."""
+[docs]
+@export
+classAlreadyInHierarchyException(AlreadyInHierarchyException,JUnitException):
+"""
+ A unit test exception raised if the element is already part of a hierarchy.
+
+ This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+ hierarchy should occur only once in the hierarchy.
+
+ .. hint::
+
+ This is usually caused by a non-None parent reference.
+ """
+
+
+
+
+[docs]
+@export
+classDuplicateTestsuiteException(DuplicateTestsuiteException,JUnitException):
+"""
+ A unit test exception raised on duplicate test suites (by name).
+
+ This exception is raised, if a child test suite with same name already exist in the test suite.
+
+ .. hint::
+
+ Test suite names need to be unique per parent element (test suite or test summary).
+ """
+
+
+
+
+[docs]
+@export
+classDuplicateTestcaseException(DuplicateTestcaseException,JUnitException):
+"""
+ A unit test exception raised on duplicate test cases (by name).
+
+ This exception is raised, if a child test case with same name already exist in the test suite.
+
+ .. hint::
+
+ Test case names need to be unique per parent element (test suite).
+ """
+[docs]
+@export
+classBase(metaclass=ExtendedType,slots=True):
+"""
+ Base-class for all test entities (test cases, test classes, test suites, ...).
+
+ It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+ hierarchy.
+
+ Every test entity has a name to identity it. It's also used in the parent's child element dictionaries to identify the
+ child. |br|
+ E.g. it's used as a test case name in the dictionary of test cases in a test class.
+ """
+
+ _parent:Nullable["Testsuite"]
+ _name:str
+
+
+[docs]
+ def__init__(self,name:str,parent:Nullable["Testsuite"]=None):
+"""
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param parent: Reference to the parent test entity.
+ :raises ValueError: If parameter 'name' is None.
+ :raises TypeError: If parameter 'name' is not a string.
+ :raises ValueError: If parameter 'name' is empty.
+ """
+ ifnameisNone:
+ raiseValueError(f"Parameter 'name' is None.")
+ elifnotisinstance(name,str):
+ ex=TypeError(f"Parameter 'name' is not of type 'str'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(name)}'.")
+ raiseex
+ elifname.strip()=="":
+ raiseValueError(f"Parameter 'name' is empty.")
+
+ self._parent=parent
+ self._name=name
+
+
+ @readonly
+ defParent(self)->Nullable["Testsuite"]:
+"""
+ Read-only property returning the reference to the parent test entity.
+
+ :return: Reference to the parent entity.
+ """
+ returnself._parent
+
+ # QUESTION: allow Parent as setter?
+
+ @readonly
+ defName(self)->str:
+"""
+ Read-only property returning the test entity's name.
+
+ :return:
+ """
+ returnself._name
+
+
+
+
+[docs]
+@export
+classBaseWithProperties(Base):
+"""
+ Base-class for all test entities supporting properties (test cases, test suites, ...).
+
+ Every test entity has fields for the test duration and number of executed assertions.
+
+ Every test entity offers an internal dictionary for properties.
+ """
+
+ _duration:Nullable[timedelta]
+ _assertionCount:Nullable[int]
+ _properties:Dict[str,Any]
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ duration:Nullable[timedelta]=None,
+ assertionCount:Nullable[int]=None,
+ parent:Nullable["Testsuite"]=None
+ ):
+"""
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param duration: Duration of the entity's execution.
+ :param assertionCount: Number of assertions within the test.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'duration' is not a timedelta.
+ :raises TypeError: If parameter 'assertionCount' is not an integer.
+ """
+ super().__init__(name,parent)
+
+ ifdurationisnotNoneandnotisinstance(duration,timedelta):
+ ex=TypeError(f"Parameter 'duration' is not of type 'timedelta'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(duration)}'.")
+ raiseex
+
+ ifassertionCountisnotNoneandnotisinstance(assertionCount,int):
+ ex=TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(assertionCount)}'.")
+ raiseex
+
+ self._duration=duration
+ self._assertionCount=assertionCount
+
+ self._properties={}
+
+
+ @readonly
+ defDuration(self)->timedelta:
+"""
+ Read-only property returning the duration of a test entity run.
+
+ .. note::
+
+ The JUnit format doesn't distinguish setup, run and teardown durations.
+
+ :return: Duration of the entity's execution.
+ """
+ returnself._duration
+
+ @readonly
+ @abstractmethod
+ defAssertionCount(self)->int:
+"""
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ .. note::
+
+ The JUnit format doesn't distinguish passed and failed assertions.
+
+ :return: Number of assertions.
+ """
+
+
+[docs]
+ def__len__(self)->int:
+"""
+ Returns the number of annotated properties.
+
+ Syntax: :pycode:`length = len(obj)`
+
+ :return: Number of annotated properties.
+ """
+ returnlen(self._properties)
+
+
+
+[docs]
+ def__getitem__(self,name:str)->Any:
+"""
+ Access a property by name.
+
+ Syntax: :pycode:`value = obj[name]`
+
+ :param name: Name if the property.
+ :return: Value of the accessed property.
+ """
+ returnself._properties[name]
+
+
+
+[docs]
+ def__setitem__(self,name:str,value:Any)->None:
+"""
+ Set the value of a property by name.
+
+ If the property doesn't exist yet, it's created.
+
+ Syntax: :pycode:`obj[name] = value`
+
+ :param name: Name of the property.
+ :param value: Value of the property.
+ """
+ self._properties[name]=value
+
+
+
+[docs]
+ def__delitem__(self,name:str)->None:
+"""
+ Delete a property by name.
+
+ Syntax: :pycode:`del obj[name]`
+
+ :param name: Name if the property.
+ """
+ delself._properties[name]
+
+
+
+[docs]
+ def__contains__(self,name:str)->bool:
+"""
+ Returns True, if a property was annotated by this name.
+
+ Syntax: :pycode:`name in obj`
+
+ :param name: Name of the property.
+ :return: True, if the property was annotated.
+ """
+ returnnameinself._properties
+
+
+
+[docs]
+ def__iter__(self)->Generator[Tuple[str,Any],None,None]:
+"""
+ Iterate all annotated properties.
+
+ Syntax: :pycode:`for name, value in obj:`
+
+ :return: A generator of property tuples (name, value).
+ """
+ yield fromself._properties.items()
+
+
+
+
+
+[docs]
+@export
+classTestcase(BaseWithProperties):
+"""
+ A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
+ Test cases are grouped by test classes in the test entity hierarchy. These are again grouped by test suites. The root
+ of the hierarchy is a test summary.
+
+ Every test case has an overall status like unknown, skipped, failed or passed.
+ """
+
+ _status:TestcaseStatus
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ duration:Nullable[timedelta]=None,
+ status:TestcaseStatus=TestcaseStatus.Unknown,
+ assertionCount:Nullable[int]=None,
+ parent:Nullable["Testclass"]=None
+ ):
+"""
+ Initializes the fields of a test case.
+
+ :param name: Name of the test entity.
+ :param duration: Duration of the entity's execution.
+ :param status: Status of the test case.
+ :param assertionCount: Number of assertions within the test.
+ :param parent: Reference to the parent test class.
+ :raises TypeError: If parameter 'parent' is not a Testsuite.
+ :raises ValueError: If parameter 'assertionCount' is not consistent.
+ """
+ ifparentisnotNone:
+ ifnotisinstance(parent,Testclass):
+ ex=TypeError(f"Parameter 'parent' is not of type 'Testclass'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testcases[name]=self
+
+ super().__init__(name,duration,assertionCount,parent)
+
+ ifnotisinstance(status,TestcaseStatus):
+ ex=TypeError(f"Parameter 'status' is not of type 'TestcaseStatus'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(status)}'.")
+ raiseex
+
+ self._status=status
+
+
+ @readonly
+ defClassname(self)->str:
+"""
+ Read-only property returning the class name of the test case.
+
+ :return: The test case's class name.
+
+ .. note::
+
+ In the JUnit format, a test case is uniquely identified by a tuple of class name and test case name. This
+ structure has been decomposed by this data model into 2 leaf-levels in the test entity hierarchy. Thus, the class
+ name is represented by its own level and instances of test classes.
+ """
+ ifself._parentisNone:
+ raiseUnittestException("Standalone Testcase instance is not linked to a Testclass.")
+ returnself._parent._name
+
+ @readonly
+ defStatus(self)->TestcaseStatus:
+"""
+ Read-only property returning the status of the test case.
+
+ :return: The test case's status.
+ """
+ returnself._status
+
+ @readonly
+ defAssertionCount(self)->int:
+"""
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ .. note::
+
+ The JUnit format doesn't distinguish passed and failed assertions.
+
+ :return: Number of assertions.
+ """
+ ifself._assertionCountisNone:
+ return0
+ returnself._assertionCount
+
+ defCopy(self)->"Testcase":
+ returnself.__class__(
+ self._name,
+ self._duration,
+ self._status,
+ self._assertionCount
+ )
+
+ defAggregate(self)->None:
+ ifself._statusisTestcaseStatus.Unknown:
+ ifself._assertionCountisNone:
+ self._status=TestcaseStatus.Passed
+ elifself._assertionCount==0:
+ self._status=TestcaseStatus.Weak
+ else:
+ self._status=TestcaseStatus.Failed
+
+ # TODO: check for setup errors
+ # TODO: check for teardown errors
+
+
+[docs]
+ @classmethod
+ defFromTestcase(cls,testcase:ut_Testcase)->"Testcase":
+"""
+ Convert a test case of the unified test entity data model to the JUnit specific data model's test case object.
+
+ :param testcase: Test case from unified data model.
+ :return: Test case from JUnit specific data model.
+ """
+ returncls(
+ testcase._name,
+ duration=testcase._testDuration,
+ status=testcase._status,
+ assertionCount=testcase._assertionCount
+ )
+
+
+ defToTestcase(self)->ut_Testcase:
+ returnut_Testcase(
+ self._name,
+ testDuration=self._duration,
+ status=self._status,
+ assertionCount=self._assertionCount,
+ # TODO: as only assertions are recorded by JUnit files, all are marked as passed
+ passedAssertionCount=self._assertionCount
+ )
+
+ defToTree(self)->Node:
+ node=Node(value=self._name)
+ node["status"]=self._status
+ node["assertionCount"]=self._assertionCount
+ node["duration"]=self._duration
+
+ returnnode
+
+
+[docs]
+@export
+classTestsuiteBase(BaseWithProperties):
+"""
+ Base-class for all test suites and for test summaries.
+
+ A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+ element in that hierarchy. While a test suite groups test classes, a test summary can only group test suites. Thus, a
+ test summary contains no test classes and test cases.
+ """
+
+ _startTime:Nullable[datetime]
+ _status:TestsuiteStatus
+
+ _tests:int
+ _skipped:int
+ _errored:int
+ _failed:int
+ _passed:int
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ startTime:Nullable[datetime]=None,
+ duration:Nullable[timedelta]=None,
+ status:TestsuiteStatus=TestsuiteStatus.Unknown,
+ parent:Nullable["Testsuite"]=None
+ ):
+"""
+ Initializes the based-class fields of a test suite or test summary.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param duration: Duration of the entity's execution.
+ :param status: Overall status of the test entity.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ """
+ ifparentisnotNone:
+ ifnotisinstance(parent,TestsuiteBase):
+ ex=TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testsuites[name]=self
+
+ super().__init__(name,duration,None,parent)
+
+ self._startTime=startTime
+ self._status=status
+ self._tests=0
+ self._skipped=0
+ self._errored=0
+ self._failed=0
+ self._passed=0
+[docs]
+@export
+classTestclass(Base):
+"""
+ A test class is a low-level element in the test entity hierarchy representing a group of tests.
+
+ Test classes contain test cases and are grouped by a test suites.
+ """
+
+ _testcases:Dict[str,"Testcase"]
+
+
+[docs]
+ def__init__(
+ self,
+ classname:str,
+ testcases:Nullable[Iterable["Testcase"]]=None,
+ parent:Nullable["Testsuite"]=None
+ ):
+"""
+ Initializes the fields of the test class.
+
+ :param classname: Classname of the test entity.
+ :param parent: Reference to the parent test suite.
+ :raises ValueError: If parameter 'classname' is None.
+ :raises TypeError: If parameter 'classname' is not a string.
+ :raises ValueError: If parameter 'classname' is empty.
+ """
+ ifparentisnotNone:
+ ifnotisinstance(parent,Testsuite):
+ ex=TypeError(f"Parameter 'parent' is not of type 'Testsuite'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testclasses[classname]=self
+
+ super().__init__(classname,parent)
+
+ self._testcases={}
+ iftestcasesisnotNone:
+ fortestcaseintestcases:
+ iftestcase._parentisnotNone:
+ raiseAlreadyInHierarchyException(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ iftestcase._nameinself._testcases:
+ raiseDuplicateTestcaseException(f"Class already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent=self
+ self._testcases[testcase._name]=testcase
+
+
+ @readonly
+ defClassname(self)->str:
+"""
+ Read-only property returning the name of the test class.
+
+ :return: The test class' name.
+ """
+ returnself._name
+
+ @readonly
+ defTestcases(self)->Dict[str,"Testcase"]:
+"""
+ Read-only property returning a reference to the internal dictionary of test cases.
+
+ :return: Reference to the dictionary of test cases.
+ """
+ returnself._testcases
+
+ @readonly
+ defTestcaseCount(self)->int:
+"""
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ returnlen(self._testcases)
+
+ @readonly
+ defAssertionCount(self)->int:
+ returnsum(tc.AssertionCountfortcinself._testcases.values())
+
+ defAddTestcase(self,testcase:"Testcase")->None:
+ iftestcase._parentisnotNone:
+ raiseValueError(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ iftestcase._nameinself._testcases:
+ raiseDuplicateTestcaseException(f"Class already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent=self
+ self._testcases[testcase._name]=testcase
+
+ defAddTestcases(self,testcases:Iterable["Testcase"])->None:
+ fortestcaseintestcases:
+ self.AddTestcase(testcase)
+
+ defToTestsuite(self)->ut_Testsuite:
+ returnut_Testsuite(
+ self._name,
+ TestsuiteKind.Class,
+ # startTime=self._startTime,
+ # totalDuration=self._duration,
+ # status=self._status,
+ testcases=(tc.ToTestcase()fortcinself._testcases.values())
+ )
+
+ defToTree(self)->Node:
+ node=Node(
+ value=self._name,
+ children=(tc.ToTree()fortcinself._testcases.values())
+ )
+
+ returnnode
+
+
+[docs]
+@export
+classTestsuite(TestsuiteBase):
+"""
+ A testsuite is a mid-level element in the test entity hierarchy representing a logical group of tests.
+
+ Test suites contain test classes and are grouped by a test summary, which is the root of the hierarchy.
+ """
+
+ _hostname:str
+ _testclasses:Dict[str,"Testclass"]
+
+
+[docs]
+ def__init__(
+ self,
+ name:str,
+ hostname:Nullable[str]=None,
+ startTime:Nullable[datetime]=None,
+ duration:Nullable[timedelta]=None,
+ status:TestsuiteStatus=TestsuiteStatus.Unknown,
+ testclasses:Nullable[Iterable["Testclass"]]=None,
+ parent:Nullable["TestsuiteSummary"]=None
+ ):
+"""
+ Initializes the fields of a test suite.
+
+ :param name: Name of the test suite.
+ :param startTime: Time when the test suite was started.
+ :param duration: duration of the entity's execution.
+ :param status: Overall status of the test suite.
+ :param parent: Reference to the parent test summary.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ :raises TypeError: If element in parameter 'testcases' is not a Testcase.
+ :raises AlreadyInHierarchyException: If a test case in parameter 'testcases' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If a test case in parameter 'testcases' is already listed (by name) in the list of test cases.
+ """
+ ifparentisnotNone:
+ ifnotisinstance(parent,TestsuiteSummary):
+ ex=TypeError(f"Parameter 'parent' is not of type 'TestsuiteSummary'.")
+ ifversion_info>=(3,11):# pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raiseex
+
+ parent._testsuites[name]=self
+
+ super().__init__(name,startTime,duration,status,parent)
+
+ self._hostname=hostname
+
+ self._testclasses={}
+ iftestclassesisnotNone:
+ fortestclassintestclasses:
+ iftestclass._parentisnotNone:
+ raiseValueError(f"Class '{testclass._name}' is already part of a testsuite hierarchy.")
+
+ iftestclass._nameinself._testclasses:
+ raiseDuplicateTestcaseException(f"Testsuite already contains a class with same name '{testclass._name}'.")
+
+ testclass._parent=self
+ self._testclasses[testclass._name]=testclass
+[docs]
+ defIterate(self,scheme:IterationScheme=IterationScheme.Default)->Generator[Union[TestsuiteType,Testcase],None,None]:
+"""
+ Iterate the test suite and its child elements according to the iteration scheme.
+
+ If no scheme is given, use the default scheme.
+
+ :param scheme: Scheme how to iterate the test suite and its child elements.
+ :returns: A generator for iterating the results filtered and in the order defined by the iteration scheme.
+ """
+ assertIterationScheme.PreOrder|IterationScheme.PostOrdernotinscheme
+
+ ifIterationScheme.PreOrderinscheme:
+ ifIterationScheme.IncludeSelf|IterationScheme.IncludeTestsuitesinscheme:
+ yieldself
+
+ ifIterationScheme.IncludeTestcasesinscheme:
+ fortestcaseinself._testclasses.values():
+ yieldtestcase
+
+ fortestclassinself._testclasses.values():
+ yield fromtestclass.Iterate(scheme|IterationScheme.IncludeSelf)
+
+ ifIterationScheme.PostOrderinscheme:
+ ifIterationScheme.IncludeTestcasesinscheme:
+ fortestcaseinself._testclasses.values():
+ yieldtestcase
+
+ ifIterationScheme.IncludeSelf|IterationScheme.IncludeTestsuitesinscheme:
+ yieldself
+
+
+
+[docs]
+ @classmethod
+ defFromTestsuite(cls,testsuite:ut_Testsuite)->"Testsuite":
+"""
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model.
+ """
+ juTestsuite=cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status=testsuite._status,
+ )
+
+ juTestsuite._tests=testsuite._tests
+ juTestsuite._skipped=testsuite._skipped
+ juTestsuite._errored=testsuite._errored
+ juTestsuite._failed=testsuite._failed
+ juTestsuite._passed=testsuite._passed
+
+ fortcintestsuite.IterateTestcases():
+ ts=tc._parent
+ iftsisNone:
+ raiseUnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname=ts._name
+ ts=ts._parent
+ whiletsisnotNoneandts._kind>TestsuiteKind.Logical:
+ classname=f"{ts._name}.{classname}"
+ ts=ts._parent
+
+ ifclassnameinjuTestsuite._testclasses:
+ juClass=juTestsuite._testclasses[classname]
+ else:
+ juClass=Testclass(classname,parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ returnjuTestsuite
+[docs]
+ defIterate(self,scheme:IterationScheme=IterationScheme.Default)->Generator[Union[Testsuite,Testcase],None,None]:
+"""
+ Iterate the test suite summary and its child elements according to the iteration scheme.
+
+ If no scheme is given, use the default scheme.
+
+ :param scheme: Scheme how to iterate the test suite summary and its child elements.
+ :returns: A generator for iterating the results filtered and in the order defined by the iteration scheme.
+ """
+ ifIterationScheme.IncludeSelf|IterationScheme.IncludeTestsuites|IterationScheme.PreOrderinscheme:
+ yieldself
+
+ fortestsuiteinself._testsuites.values():
+ yield fromtestsuite.IterateTestsuites(scheme|IterationScheme.IncludeSelf)
+
+ ifIterationScheme.IncludeSelf|IterationScheme.IncludeTestsuites|IterationScheme.PostOrderinscheme:
+ yieldself
+
+
+
+[docs]
+ @classmethod
+ defFromTestsuiteSummary(cls,testsuiteSummary:ut_TestsuiteSummary)->"TestsuiteSummary":
+"""
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model.
+ """
+ returncls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite)fortestsuiteintestsuiteSummary._testsuites.values())
+ )
+
+
+
+[docs]
+ defToTestsuiteSummary(self)->ut_TestsuiteSummary:
+"""
+ Convert this test suite summary a new test suite summary of the unified data model.
+
+ All fields are copied to the new instance. Child elements like test suites are copied recursively.
+
+ :return: A test suite summary of the unified test entity data model.
+ """
+ returnut_TestsuiteSummary(
+ self._name,
+ startTime=self._startTime,
+ totalDuration=self._duration,
+ status=self._status,
+ testsuites=(testsuite.ToTestsuite()fortestsuiteinself._testsuites.values())
+ )
+[docs]
+ defAnalyze(self)->None:
+"""
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`.
+
+ The used XML schema definition is generic to support "any" dialect.
+ """
+ xmlSchemaFile="Any-JUnit.xsd"
+ self._Analyze(xmlSchemaFile)
+
+
+ def_Analyze(self,xmlSchemaFile:str)->None:
+ ifnotself._path.exists():
+ raiseUnittestException(f"JUnit XML file '{self._path}' does not exist.") \
+ fromFileNotFoundError(f"File '{self._path}' not found.")
+
+ startAnalysis=perf_counter_ns()
+ try:
+ xmlSchemaResourceFile=getResourceFile(Resources,xmlSchemaFile)
+ exceptToolingExceptionasex:
+ raiseUnittestException(f"Couldn't locate XML Schema '{xmlSchemaFile}' in package resources.")fromex
+
+ try:
+ schemaParser=XMLParser(ns_clean=True)
+ schemaRoot=parse(xmlSchemaResourceFile,schemaParser)
+ exceptXMLSyntaxErrorasex:
+ raiseUnittestException(f"XML Syntax Error while parsing XML Schema '{xmlSchemaFile}'.")fromex
+
+ try:
+ junitSchema=XMLSchema(schemaRoot)
+ exceptXMLSchemaParseErrorasex:
+ raiseUnittestException(f"Error while parsing XML Schema '{xmlSchemaFile}'.")
+
+ try:
+ junitParser=XMLParser(schema=junitSchema,ns_clean=True)
+ junitDocument=parse(self._path,parser=junitParser)
+
+ self._xmlDocument=junitDocument
+ exceptXMLSyntaxErrorasex:
+ ifversion_info>=(3,11):# pragma: no cover
+ forlogEntryinjunitParser.error_log:
+ ex.add_note(str(logEntry))
+ raiseUnittestException(f"XML syntax or validation error for '{self._path}' using XSD schema '{xmlSchemaResourceFile}'.")fromex
+ exceptExceptionasex:
+ raiseUnittestException(f"Couldn't open '{self._path}'.")fromex
+
+ endAnalysis=perf_counter_ns()
+ self._analysisDuration=(endAnalysis-startAnalysis)/1e9
+
+
+[docs]
+ defWrite(self,path:Nullable[Path]=None,overwrite:bool=False,regenerate:bool=False)->None:
+"""
+ Write the data model as XML into a file adhering to the Any JUnit dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ ifpathisNone:
+ path=self._path
+
+ ifnotoverwriteandpath.exists():
+ raiseUnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ fromFileExistsError(f"File '{path}' already exists.")
+
+ ifregenerate:
+ self.Generate(overwrite=True)
+
+ ifself._xmlDocumentisNone:
+ ex=UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raiseex
+
+ try:
+ withpath.open("wb")asfile:
+ file.write(tostring(self._xmlDocument,encoding="utf-8",xml_declaration=True,pretty_print=True))
+ exceptExceptionasex:
+ raiseUnittestException(f"JUnit XML file '{path}' can not be written.")fromex
+
+
+
+[docs]
+ defConvert(self)->None:
+"""
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ ifself._xmlDocumentisNone:
+ ex=UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raiseex
+
+ startConversion=perf_counter_ns()
+ rootElement:_Element=self._xmlDocument.getroot()
+
+ self._name=self._ConvertName(rootElement,optional=True)
+ self._startTime=self._ConvertTimestamp(rootElement,optional=True)
+ self._duration=self._ConvertTime(rootElement,optional=True)
+
+ ifFalse:# self._readerMode is JUnitReaderMode.
+ self._tests=self._ConvertTests(testsuitesNode)
+ self._skipped=self._ConvertSkipped(testsuitesNode)
+ self._errored=self._ConvertErrors(testsuitesNode)
+ self._failed=self._ConvertFailures(testsuitesNode)
+ self._assertionCount=self._ConvertAssertions(testsuitesNode)
+
+ forrootNodeinrootElement.iterchildren(tag="testsuite"):# type: _Element
+ self._ConvertTestsuite(self,rootNode)
+
+ ifTrue:# self._readerMode is JUnitReaderMode.
+ self.Aggregate()
+
+ endConversation=perf_counter_ns()
+ self._modelConversion=(endConversation-startConversion)/1e9
+
+
+
+[docs]
+ def_ConvertName(self,element:_Element,default:str="root",optional:bool=True)->str:
+"""
+ Convert the ``name`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``name`` attribute.
+ :param default: The default value, if no ``name`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``name`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``name`` attribute exists on the given element node.
+ """
+ if"name"inelement.attrib:
+ returnelement.attrib["name"]
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'name' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertTimestamp(self,element:_Element,optional:bool=True)->Nullable[datetime]:
+"""
+ Convert the ``timestamp`` attribute from an XML element node to a datetime.
+
+ :param element: The XML element node with a ``timestamp`` attribute.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``timestamp`` attribute's content if found, otherwise ``None``.
+ :raises UnittestException: If optional is false and no ``timestamp`` attribute exists on the given element node.
+ """
+ if"timestamp"inelement.attrib:
+ timestamp=element.attrib["timestamp"]
+ returndatetime.fromisoformat(timestamp)
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'timestamp' not found in tag '{element.tag}'.")
+ else:
+ returnNone
+
+
+
+[docs]
+ def_ConvertTime(self,element:_Element,optional:bool=True)->Nullable[timedelta]:
+"""
+ Convert the ``time`` attribute from an XML element node to a timedelta.
+
+ :param element: The XML element node with a ``time`` attribute.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``time`` attribute's content if found, otherwise ``None``.
+ :raises UnittestException: If optional is false and no ``time`` attribute exists on the given element node.
+ """
+ if"time"inelement.attrib:
+ time=element.attrib["time"]
+ returntimedelta(seconds=float(time))
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'time' not found in tag '{element.tag}'.")
+ else:
+ returnNone
+
+
+
+[docs]
+ def_ConvertHostname(self,element:_Element,default:str="localhost",optional:bool=True)->str:
+"""
+ Convert the ``hostname`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``hostname`` attribute.
+ :param default: The default value, if no ``hostname`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``hostname`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``hostname`` attribute exists on the given element node.
+ """
+ if"hostname"inelement.attrib:
+ returnelement.attrib["hostname"]
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'hostname' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertClassname(self,element:_Element)->str:
+"""
+ Convert the ``classname`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``classname`` attribute.
+ :return: The ``classname`` attribute's content.
+ :raises UnittestException: If no ``classname`` attribute exists on the given element node.
+ """
+ if"classname"inelement.attrib:
+ returnelement.attrib["classname"]
+ else:
+ raiseUnittestException(f"Required parameter 'classname' not found in tag '{element.tag}'.")
+
+
+
+[docs]
+ def_ConvertTests(self,element:_Element,default:Nullable[int]=None,optional:bool=True)->Nullable[int]:
+"""
+ Convert the ``tests`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``tests`` attribute.
+ :param default: The default value, if no ``tests`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``tests`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``tests`` attribute exists on the given element node.
+ """
+ if"tests"inelement.attrib:
+ returnint(element.attrib["tests"])
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'tests' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertSkipped(self,element:_Element,default:Nullable[int]=None,optional:bool=True)->Nullable[int]:
+"""
+ Convert the ``skipped`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``skipped`` attribute.
+ :param default: The default value, if no ``skipped`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``skipped`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``skipped`` attribute exists on the given element node.
+ """
+ if"skipped"inelement.attrib:
+ returnint(element.attrib["skipped"])
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'skipped' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertErrors(self,element:_Element,default:Nullable[int]=None,optional:bool=True)->Nullable[int]:
+"""
+ Convert the ``errors`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``errors`` attribute.
+ :param default: The default value, if no ``errors`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``errors`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``errors`` attribute exists on the given element node.
+ """
+ if"errors"inelement.attrib:
+ returnint(element.attrib["errors"])
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'errors' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertFailures(self,element:_Element,default:Nullable[int]=None,optional:bool=True)->Nullable[int]:
+"""
+ Convert the ``failures`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``failures`` attribute.
+ :param default: The default value, if no ``failures`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``failures`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``failures`` attribute exists on the given element node.
+ """
+ if"failures"inelement.attrib:
+ returnint(element.attrib["failures"])
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'failures' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertAssertions(self,element:_Element,default:Nullable[int]=None,optional:bool=True)->Nullable[int]:
+"""
+ Convert the ``assertions`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``assertions`` attribute.
+ :param default: The default value, if no ``assertions`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``assertions`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``assertions`` attribute exists on the given element node.
+ """
+ if"assertions"inelement.attrib:
+ returnint(element.attrib["assertions"])
+ elifnotoptional:
+ raiseUnittestException(f"Required parameter 'assertions' not found in tag '{element.tag}'.")
+ else:
+ returndefault
+
+
+
+[docs]
+ def_ConvertTestsuite(self,parent:TestsuiteSummary,testsuitesNode:_Element)->None:
+"""
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite=self._TESTSUITE(
+ self._ConvertName(testsuitesNode,optional=False),
+ self._ConvertHostname(testsuitesNode,optional=True),
+ self._ConvertTimestamp(testsuitesNode,optional=True),
+ self._ConvertTime(testsuitesNode,optional=True),
+ parent=parent
+ )
+
+ ifFalse:# self._readerMode is JUnitReaderMode.
+ self._tests=self._ConvertTests(testsuitesNode)
+ self._skipped=self._ConvertSkipped(testsuitesNode)
+ self._errored=self._ConvertErrors(testsuitesNode)
+ self._failed=self._ConvertFailures(testsuitesNode)
+ self._assertionCount=self._ConvertAssertions(testsuitesNode)
+
+ self._ConvertTestsuiteChildren(testsuitesNode,newTestsuite)
+[docs]
+ def_ConvertTestcase(self,parent:Testsuite,testcaseNode:_Element)->None:
+"""
+ Convert the XML data structure of a ``<testcase>`` to a test case.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite as a parent element in the test entity hierarchy.
+ :param testcaseNode: The current XML element node representing a test case.
+ """
+ className=self._ConvertClassname(testcaseNode)
+ testclass=self._FindOrCreateTestclass(parent,className)
+
+ newTestcase=self._TESTCASE(
+ self._ConvertName(testcaseNode,optional=False),
+ self._ConvertTime(testcaseNode,optional=False),
+ assertionCount=self._ConvertAssertions(testcaseNode),
+ parent=testclass
+ )
+
+ self._ConvertTestcaseChildren(testcaseNode,newTestcase)
+[docs]
+ defGenerate(self,overwrite:bool=False)->None:
+"""
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuites>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ ifnotoverwriteandself._xmlDocumentisnotNone:
+ raiseUnittestException(f"Internal XML document is populated with data.")
+
+ rootElement=Element("testsuites")
+ rootElement.attrib["name"]=self._name
+ ifself._startTimeisnotNone:
+ rootElement.attrib["timestamp"]=f"{self._startTime.isoformat()}"
+ ifself._durationisnotNone:
+ rootElement.attrib["time"]=f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"]=str(self._tests)
+ rootElement.attrib["failures"]=str(self._failed)
+ rootElement.attrib["errors"]=str(self._errored)
+ rootElement.attrib["skipped"]=str(self._skipped)
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+
+ self._xmlDocument=ElementTree(rootElement)
+
+ fortestsuiteinself._testsuites.values():
+ self._GenerateTestsuite(testsuite,rootElement)
+
+
+
+[docs]
+ def_GenerateTestsuite(self,testsuite:Testsuite,parentElement:_Element)->None:
+"""
+ Generate the internal XML data structure for a test suite.
+
+ This method generates the XML element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param testsuite: The test suite to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testsuiteElement=SubElement(parentElement,"testsuite")
+ testsuiteElement.attrib["name"]=testsuite._name
+ iftestsuite._startTimeisnotNone:
+ testsuiteElement.attrib["timestamp"]=f"{testsuite._startTime.isoformat()}"
+ iftestsuite._durationisnotNone:
+ testsuiteElement.attrib["time"]=f"{testsuite._duration.total_seconds():.6f}"
+ testsuiteElement.attrib["tests"]=str(testsuite._tests)
+ testsuiteElement.attrib["failures"]=str(testsuite._failed)
+ testsuiteElement.attrib["errors"]=str(testsuite._errored)
+ testsuiteElement.attrib["skipped"]=str(testsuite._skipped)
+ # if testsuite._assertionCount is not None:
+ # testsuiteElement.attrib["assertions"] = f"{testsuite._assertionCount}"
+ iftestsuite._hostnameisnotNone:
+ testsuiteElement.attrib["hostname"]=testsuite._hostname
+
+ fortestclassintestsuite._testclasses.values():
+ fortcintestclass._testcases.values():
+ self._GenerateTestcase(tc,testsuiteElement)
+
+
+
+[docs]
+ def_GenerateTestcase(self,testcase:Testcase,parentElement:_Element)->None:
+"""
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement=SubElement(parentElement,"testcase")
+ iftestcase.ClassnameisnotNone:
+ testcaseElement.attrib["classname"]=testcase.Classname
+ testcaseElement.attrib["name"]=testcase._name
+ iftestcase._durationisnotNone:
+ testcaseElement.attrib["time"]=f"{testcase._duration.total_seconds():.6f}"
+ iftestcase._assertionCountisnotNone:
+ testcaseElement.attrib["assertions"]=f"{testcase._assertionCount}"
+
+ iftestcase._statusisTestcaseStatus.Passed:
+ pass
+ eliftestcase._statusisTestcaseStatus.Failed:
+ failureElement=SubElement(testcaseElement,"failure")
+ eliftestcase._statusisTestcaseStatus.Skipped:
+ skippedElement=SubElement(testcaseElement,"skipped")
+ else:
+ errorElement=SubElement(testcaseElement,"error")
275ifassertionCountisnotNoneandnotisinstance(assertionCount,int):275 ↛ 276line 275 didn't jump to line 276 because the condition on line 275 was never true
+
276ex=TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
346iftestDurationisnotNoneandnotisinstance(testDuration,timedelta):346 ↛ 347line 346 didn't jump to line 347 because the condition on line 346 was never true
+
347ex=TypeError(f"Parameter 'testDuration' is not of type 'timedelta'.")
352ifsetupDurationisnotNoneandnotisinstance(setupDuration,timedelta):352 ↛ 353line 352 didn't jump to line 353 because the condition on line 352 was never true
+
353ex=TypeError(f"Parameter 'setupDuration' is not of type 'timedelta'.")
358ifteardownDurationisnotNoneandnotisinstance(teardownDuration,timedelta):358 ↛ 359line 358 didn't jump to line 359 because the condition on line 358 was never true
+
359ex=TypeError(f"Parameter 'teardownDuration' is not of type 'timedelta'.")
364iftotalDurationisnotNoneandnotisinstance(totalDuration,timedelta):364 ↛ 365line 364 didn't jump to line 365 because the condition on line 364 was never true
+
365ex=TypeError(f"Parameter 'totalDuration' is not of type 'timedelta'.")
434ifkeyValuePairsisnotNoneandnotisinstance(keyValuePairs,Mapping):434 ↛ 435line 434 didn't jump to line 435 because the condition on line 434 was never true
+
435ex=TypeError(f"Parameter 'keyValuePairs' is not a mapping.")
695ifassertionCountisnotNoneandnotisinstance(assertionCount,int):695 ↛ 696line 695 didn't jump to line 696 because the condition on line 695 was never true
+
696ex=TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
701iffailedAssertionCountisnotNoneandnotisinstance(failedAssertionCount,int):701 ↛ 702line 701 didn't jump to line 702 because the condition on line 701 was never true
+
702ex=TypeError(f"Parameter 'failedAssertionCount' is not of type 'int'.")
707ifpassedAssertionCountisnotNoneandnotisinstance(passedAssertionCount,int):707 ↛ 708line 707 didn't jump to line 708 because the condition on line 707 was never true
+
708ex=TypeError(f"Parameter 'passedAssertionCount' is not of type 'int'.")
1613ifIterationScheme.IncludeSelf|IterationScheme.IncludeTestsuites|IterationScheme.PostOrderinscheme:1613 ↛ 1614line 1613 didn't jump to line 1614 because the condition on line 1613 was never true
143 This base-class for :class:`ModuleCoverage` and :class:`PackageCoverage` represents an extended set of documentation coverage metrics, especially with aggregated metrics.
459ifnotdirectory.exists():459 ↛ 460line 459 didn't jump to line 460 because the condition on line 459 was never true
+
460raiseDocStrCoverageError(f"Package source directory '{directory}' does not exist.")fromFileNotFoundError(f"Directory '{directory}' does not exist.")
519ifcurrentCoverageObject._uncovered!=currentCoverageObject._expected-currentCoverageObject._covered:519 ↛ 520line 519 didn't jump to line 520 because the condition on line 519 was never true
This package provides abstract data models and specific implementations for report formats. The supported report formats
+are commonly used for any programming language or have a specifc context with Electronic Design Automation (EDA) tools.
+Examples are unit test summaries (like Ant JUnit XML), code coverage (like Cobertura) and documentation coverage reports.
+
While the data models and file format implementations can be used as a library, a CLI program pyedaa-report will be
+provided too. It allows reading, converting, concatenating, merging, transforming and writing report files.
+
+
Roadmap
+
It’s also planned to support console outputs from simulators and synthesis/implementation tools to create structured
+logs and reports for filtering and data extraction.
Code coverage measures used and unused code lines, statements, branches, etc. Depending on the programming
+language this is measured by instrumenting the code/binary and running the program, it’s test cases or simulating
+the code. In generate code coverage is a measure of test coverage. Unused code is not (yet) covered by tests.
+
The code coverage metric in percent is a ratio of used code versus all possibly usable code. A coverage of <100%
+indicates unused code. This can be dead code (unreachable) or untested code (⇒ needs more test cases).
Documentation coverage measures the presence of code documentation. It primarily counts for public language
+entities like publicly visible constants and variables, parameters, types, functions, methods, classes, modules,
+packages, etc. The documentation goal depends on the used coverage collection tool’s settings. E.g. usually,
+private language entities are not required to be documented.
+
The documentation coverage metric in percent is a ratio of documented language entity versus all documentation
+worthy langauge entities. A coverage of <100% indicates undocumented code.
Results of (unit) tests (also regression tests) are collected in machine readable summary files. Test cases are
+usually grouped by one or more test suites. Besides the test’s result (passed, failed, skipped, …) also the
+test’s outputs and durations are collected. Results can be visualized as a expandable tree structure.
+
The total number of testcases indicates the spend effort in testing and applying many test vectors. In combination
+with code coverage, it can be judged if the code has untested sections.
Both GitHub and GitLab do provide features for displaying CI results through their web GUIs. Although they are not rich
+enough for displaying all the details, OSVR generators can provide stripped down file formats matching some of the
+supported readers; similarly to the JSON and xUnit outputs provided by VUnit.
It would be interesting to have a vendor agnostic tool for visualizing reports locally and/or in self-hosted services.
+Since XML, JSON or YAML are used, web technologies (HTML + CSS + JavaScript) feel like a sensible choice. Generating an
+static page which can be hosted on GitHub Pages or GitLab Pages allows granular analysis of CI results, while also being
+usable locally. There are several simple and not-so-simple solutions available for xUnit files:
As a complement, extending pyucis-viewer might be evaluated, for providing
+a Qt based solution. pyucis-viewer currently provides a simple bar-chart viewer for coverage data read via pyucis.
Although there is no official feature for using the GitHub Checks
+API, there are some community actions for e.g. analysing xUnit files: publish-unit-test-results. There are also multiple bindings in JavaScript, Python or golang for
+interacting with GitHub’s API.
On top of visualizing individual reports or sets of reports at one point in time, tracking the evolution of certain
+metrics throughout the development of a project can provide valuable insight. GitLab does have a built-in Prometheus monitoring system and Grafana can be optionally used as a dashboard:
+docs.gitlab.com: Grafana Dashboard Service. Therefore, it
+would be useful to send OSVR reports to either Prometheus or some other temporal database (say Graphite, InfluxDB, etc.).
xUnit (every unit test is a cover point that has a binary pass|fail).
+VUnit, cocotb, fsva and others can generate xUnit reports of unit testing suites.
+
When PSL is used, GHDL can generate a JSON report of cover and assert statements: --psl-report.
+
OSVVM has an internal coverge database format.
+
There is an specification by Accellera, Mentor Graphics and Cadence named Unified Coverage Interoperability Standard (UCIS) and a matching Unified Coverage Database (UCDB).
VUnit has built-in support for generating xUnit (XML) reports. In fact,
+VUnit’s name comes from VHDL unit testing framework (see Wikipedia: List of unit testing frameworks).
+CLI option -x allows specifying the target file name. Two different formats are supported: Jenkins
+(JUnit) and Bamboo. JUnit is
+also supported on GitLab CI: docs.gitlab.com: Unit test reports.
+Python’s unittest (and, therefore, pytest) was originally inspired by JUnit, so it has a similar flavor as unit testing
+frameworks in other languages. Moreover, there is junitparser, a Python tool
+for manipulating xUnit XML files.
+
Therefore, by using VUnit’s simulator interface and test runner infrastructure, it is already possible to generate fine
+grained reports in a standard format. This might be useful for users of OSVVM and/or UVVM, which don’t have an
+equivalent feature.
+
Cocotb can also generate xUnit reports, independently from VUnit. See docs.cocotb.org: COCOTB_RESULTS_FILE.
+Precisely, this is related to the duplicated test/regression management features in both frameworks. At the moment,
+users are expected to handle them independently when mixed (HDL + cocotb) testsuites are run. However, there is work in
+progress for hopefully unifying them automatically (through some post-simulation helper hook). Anyway, while generated
+independently, the OSVR core can be used for aggregating them.
+
+
Note
+
In the JUnit XML format, the result of each test is only explicitly provided in case of failure, error or skip.
+Therefore, the absence of result indicates a passed test case.
As explained in --psl-report, “for each PSL cover and assert statements, the name, source location and whether it passed or failed is reported” by GHDL in a JSON format. Therefore, it should be trivial
+to import these reports in OSVR similarly to how xUnit reports are handled.
OSVVM has a non-trivial built-in database format for the advanced functional coverage features provided by
+CoveragePkg (see OSVVM/Documentation: CoveragePkg_*.pdf). There is work in progress with developers of OSVVM for evaluating how
+to export it to some standard format, such as xUnit, UCB, or some other XML/JSON/YAML format.
+
The main constraint for displaying combined results of multidimensional coverage analysis is that xUnit is expected to have a single level of hierarchy (suites and tests). Hence, unlike previous projects, OSVVM might need some more elaborated format.
Unified Coverage Database (UCDB) is one of the components of the Unified Coverage Interoperability Standard (UCIS)
+developed by Accellera, Mentor Graphics and Cadence. The UCDB is used by Siemens’ tools for tracking results, and they
+have a GUI module for browsing them. At first sight, UCDB/UCIS are complex and not easy to work with, however, most of
+the potential result types are already covered by the specification (see Unified Coverage Interoperability Standard Version
+and OSVVM Forums: Cover group and Mentor UCDB).
+See also OSVVM Forums: UCIS / UCDB.
+Fortunately, there is an open source Python package that provides an API to UCIS data (fvutils/pyucis)
+as well as an open source Qt based GUI (fvutils/pyucis-viewer). pyucis
+can write coverage data in UCIS XML-interchange format and to mentor UCDB via the UCIS library provided by Questa.
+Hence, it might be possible to dump results from open source frameworks/methodologies/tools to UCDB for reusing Siemens’
+or fvutils’ GUIs, or vice versa.
+
+
Note
+
From an open source community perspective, it feels more sensible to dump content from UCDB to an open source
+XML/JSON/YAML format specification. However, as far as we are aware, such FLOSS specification adapted to hardware
+designs does not exist yet. Moreover, the most used HDL languages are neither open source. Hence, although not ideal,
+using UCDB wouldn’t be disruptive in this regard. Should you know about any open source alternative, or if you
+represent Accelera, Siemens’ and/or Cadence and want to open source UCDB/UCIS, please let us know!
While all tools provide feedback by logging into terminal, many of them do also provide results through some report file
+format.
+However, there is no standard/universal report format which can gather all the diferent types of results that EDA tools
+can provide.
+pyEDAA.Reports is a proposal for achieving it.
+
As shown in the figure below, the main purpose is to allow reusing existing frontends (such as GitHub Checks,
+Gitlab job artifacts and unit test reports,
+and/or Grafana) from software testing/verification, rather than reinventing a solution from
+scratch. By the same token, we would like to use coverage and test report formats which are compatible with the existing
+open source ecosystem used by software people.
The most basic functionality is adding a hierarchy level on top of xUnit, for aggregating multiple xUnit reports
+corresponding to the same design/project.
+That additional hierarchy might be encoded as an additional field in the XML, or by prepending suite names with specific
+keywords.
+On top of that, some content in hardware project reports need some more elaborated formats.
+In the following subsections, each report type is analysed.
Several open source tools (such as GCC or PyPI’s coverage package) produce
+line coverage results in gcov format. Moreover, utilities exist for generating
+reports from gcov files. For instance gcovr can produce html, xml
+(Cobertura), sonarqube and json. Furthermore, some HDL tools, such
+as GHDL with GCC backend, can generate gcov results too.
Some synthesis and all implementation tools do provide reports about area/resource usage and estimated maximum clock
+frequency. However, most tools do print tables in logs. Parsing them is not complex (see YosysHQ/arachne-pnr#78), but it needs to be done ad-hoc. Some vendors, such as Vivado, do also
+report resource in text logs, but do allow to export them as *.xls (say CSV) files.
+
Edalize supports parsing/reading reports from some EDA tools:
+
+
edalize.reporting
+
edalize.vivado_reporting
+
edalize.quartus_reporting
+
edalize.ise_reporting
+
+
SymbiFlow/fpga-tool-perf does also support extracting results from
+Vivado, Yosys, Verilog to Routing and Nextpnr.
+Moreover, results are gathered in a Collab Dashboard: Symbiflow Dashboard GCS.
+
mattvenn/logLUTs allows parsing yosys and nextpnr logfiles to then plot LUT,
+flip-flop and maximum frequency stats.
+
+
Note
+
The fields in resource usage reports can be provided as absolute values or relative to the capacity of the device.
+Therefore, fields in resource reports of OSVR can and should match the ones in hdl/constraints: template/device.info.yml.
+Since both OSVR and the device template in hdl/constraints are subject to change yet, we should make them similar to
+existing solutions.
xUnit report files (XML) typically provide the relevant raw log output together with the errored, failed or skipped
+result.
+However, most verification frameworks, tools and methodologies do have more granular information about each entry.
+At least, the severity level is a built-in feature in VHDL, and several projects do provide additional logging utilities
+with further severity levels or failure reasons.
+For instance, VUnit supports custom logging levels, and can export rich logs to CSV files.
+Moreover, pyIPCMI includes vendor log processing features for classifying and
+optionally filtering the logs.
+Therefore, it would be interesting to support preserving the semantic information (at least the severity or specific
+vendor error/report code), in the extended xUnit report format used in OSVR.
+On top of that, librecores/eda-log-parser supports parsing logs from
+Verilator and Vivado, along with generating custom log entries to be used in CI systems/services, such as Azure or
+GitHub Actions.
Industries developing systems for critical applications do typically require tracking specification requirements through
+the developement of the products.
+See, for instance, Using GitLab for ISO 26262-6:2018 - Product development at the software level.
+Hence, it is very valuable to annotate tests with requirements, and then cross-reference tests and CI runs with those.
+In the open source ecosystem, some projects create test cases for each reported MWE through a GitHub/GitLab issue.
+Therefore, in such contexts the issue numbers, tags or milestones might be considered requirements to be tracked.
+
There is an example by Lars Asplund (from VUnit), for illustrating the usage of VUnit attributes for tracking requirements:
+LarsAsplund/vunit_attributes.
+It provides requirement to attribute mapping through the --export-json option, which is a richer format than the
+xUnit produced with -x.
+In the example, additional analysis features are provided through a requirement coverage analysis script:
+analyze_requirement_coverage.py.
+The list of requirements is defined in a CSV file.
+
Precisely, field Metadata proposed in the OSVR Testcase class is expected to contain data such as the attributes.
+That is, to integrate VUnit’s attribute tracking, with other frameworks which might provide similar features.
Besides the total number of coverable items, it distinguishes items as excluded, ignored, and expected.
+Expected items are further distinguished into covered and uncovered items.
+If no item is expected, then coverage is always 100 %.
This base-class for ModuleCoverage and PackageCoverage represents an extended set of documentation coverage metrics, especially with aggregated metrics.
+
As inherited from Coverage, it provides the total number of coverable items, which are distinguished into
+excluded, ignored, and expected items.
+Expected items are further distinguished into covered and uncovered items.
+If no item is expected, then coverage and aggregated coverage are always 100 %.
+
In addition, all previously mentioned metrics are collected as aggregated…, too.
name: the name of the member
+start: the initial start value or None
+count: the number of existing members
+last_values: the last value assigned or None
The pyEDAA.Reports.Unittesting.JUnit package implements a hierarchy of test entities for the JUnit unit testing summary
+file format (XML format). This test entity hierarchy is not derived from pyEDAA.Reports.Unittesting, because it
+doesn’t match the unified data model. Nonetheless, both data models can be converted to each other. In addition, derived
+data models are provided for the many dialects of that XML file format. See the list modules in this package for the
+implemented dialects.
+
The test entity hierarchy consists of test cases, test classes, test suites and a test summary. Test cases are the leaf
+elements in the hierarchy and represent an individual test run. Next, test classes group test cases, because the
+original Ant + JUnit format groups test cases (Java methods) in a Java class. Next, test suites are used to group
+multiple test classes. Finally, the root element is a test summary. When such a summary is stored in a file format like
+Ant + JUnit4 XML, a file format specific document is derived from a summary class.
A unit test exception raised if the element is already part of a hierarchy.
+
This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+hierarchy should occur only once in the hierarchy.
+
+
Hint
+
This is usually caused by a non-None parent reference.
name: the name of the member
+start: the initial start value or None
+count: the number of existing members
+last_values: the last value assigned or None
Base-class for all test entities (test cases, test classes, test suites, …).
+
It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+hierarchy.
+
Every test entity has a name to identity it. It’s also used in the parent’s child element dictionaries to identify the
+child.
+E.g. it’s used as a test case name in the dictionary of test cases in a test class.
A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
Test cases are grouped by test classes in the test entity hierarchy. These are again grouped by test suites. The root
+of the hierarchy is a test summary.
+
Every test case has an overall status like unknown, skipped, failed or passed.
Read-only property returning the class name of the test case.
+
+
Returns:
+
The test case’s class name.
+
+
+
+
Note
+
In the JUnit format, a test case is uniquely identified by a tuple of class name and test case name. This
+structure has been decomposed by this data model into 2 leaf-levels in the test entity hierarchy. Thus, the class
+name is represented by its own level and instances of test classes.
Base-class for all test suites and for test summaries.
+
A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+element in that hierarchy. While a test suite groups test classes, a test summary can only group test suites. Thus, a
+test summary contains no test classes and test cases.
This includes usually the duration to validate and parse the file format, but it excludes the time to convert the
+content to the test entity hierarchy.
This includes usually the duration to convert the document’s content to the test entity hierarchy. It might also
+include the duration to (re-)aggregate all states and statistics in the hierarchy.
The pyEDAA.Reports.Unittesting package implements a hierarchy of test entities. These are test cases, test suites and a
+test summary provided as a class hierarchy. Test cases are the leaf elements in the hierarchy and abstract an
+individual test run. Test suites are used to group multiple test cases or other test suites. The root element is a test
+summary. When such a summary is stored in a file format like Ant + JUnit4 XML, a file format specific document is
+derived from a summary class.
A unit test exception raised if the element is already part of a hierarchy.
+
This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+hierarchy should occur only once in the hierarchy.
+
+
Hint
+
This is usually caused by a non-None parent reference.
name: the name of the member
+start: the initial start value or None
+count: the number of existing members
+last_values: the last value assigned or None
name: the name of the member
+start: the initial start value or None
+count: the number of existing members
+last_values: the last value assigned or None
Length of bytes object to use. An OverflowError is raised if the
+integer is not representable with the given number of bytes. Default
+is length 1.
+
+
byteorder
The byte order used to represent the integer. If byteorder is ‘big’,
+the most significant byte is at the beginning of the byte array. If
+byteorder is ‘little’, the most significant byte is at the end of the
+byte array. To request the native byte order of the host system, use
+sys.byteorder as the byte order value. Default is to use ‘big’.
+
+
signed
Determines whether two’s complement is used to represent the integer.
+If signed is False and a negative integer is given, an OverflowError
+is raised.
Return the integer represented by the given array of bytes.
+
+
bytes
Holds the array of bytes to convert. The argument must either
+support the buffer protocol or be an iterable object producing bytes.
+Bytes and bytearray are examples of built-in objects that support the
+buffer protocol.
+
+
byteorder
The byte order used to represent the integer. If byteorder is ‘big’,
+the most significant byte is at the beginning of the byte array. If
+byteorder is ‘little’, the most significant byte is at the end of the
+byte array. To request the native byte order of the host system, use
+sys.byteorder as the byte order value. Default is to use ‘big’.
+
+
signed
Indicates whether two’s complement is used to represent the integer.
A flag enumeration for selecting the test suite iteration scheme.
+
When a test entity hierarchy is (recursively) iterated, this iteration scheme describes how to iterate the hierarchy
+and what elements to return as a result.
name: the name of the member
+start: the initial start value or None
+count: the number of existing members
+last_values: the last value assigned or None
Base-class for all test entities (test cases, test suites, …).
+
It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+hierarchy.
+
Every test entity has a name to identity it. It’s also used in the parent’s child element dictionaries to identify the
+child.
+E.g. it’s used as a test case name in the dictionary of test cases in a test suite.
+
Every test entity has fields for time tracking. If known, a start time and a test duration can be set. For more
+details, a setup duration and teardown duration can be added. All durations are summed up in a total duration field.
+
As tests can have warnings and errors or even fail, these messages are counted and aggregated in the test entity
+hierarchy.
+
Every test entity offers an internal dictionary for annotations.
+This feature is for example used by Ant + JUnit4’s XML property fields.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
Test cases are grouped by test suites in the test entity hierarchy. The root of the hierarchy is a test summary.
+
Every test case has an overall status like unknown, skipped, failed or passed.
+
In addition to all features from its base-class, test cases provide additional statistics for passed and failed
+assertions (checks) as well as a sum thereof.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
Base-class for all test suites and for test summaries.
+
A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+element in that hierarchy. While a test suite groups other test suites and test cases, a test summary can only group
+test suites. Thus, a test summary contains no test cases.
Read-only property returning the kind of the test suite.
+
Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+language construct.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
At least, parameterizing a generic class is the main thing this
+method does. For example, for some generic class Foo, this is called
+when we do Foo[int] - there, with cls=Foo and params=int.
+
However, note that this method is also called when defining generic
+classes in the first place with class Foo[T]: ….
A testsuite is a mid-level element in the test entity hierarchy representing a group of tests.
+
Test suites contain test cases and optionally other test suites. Test suites can be grouped by test suites to form a
+hierarchy of test entities. The root of the hierarchy is a test summary.
Read-only property returning the kind of the test suite.
+
Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+language construct.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
At least, parameterizing a generic class is the main thing this
+method does. For example, for some generic class Foo, this is called
+when we do Foo[int] - there, with cls=Foo and params=int.
+
However, note that this method is also called when defining generic
+classes in the first place with class Foo[T]: ….
Read-only property returning the kind of the test suite.
+
Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+language construct.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
At least, parameterizing a generic class is the main thing this
+method does. For example, for some generic class Foo, this is called
+when we do Foo[int] - there, with cls=Foo and params=int.
+
However, note that this method is also called when defining generic
+classes in the first place with class Foo[T]: ….
This includes usually the duration to validate and parse the file format, but it excludes the time to convert the
+content to the test entity hierarchy.
This includes usually the duration to convert the document’s content to the test entity hierarchy. It might also
+include the duration to (re-)aggregate all states and statistics in the hierarchy.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
Read-only property returning the kind of the test suite.
+
Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+language construct.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
At least, parameterizing a generic class is the main thing this
+method does. For example, for some generic class Foo, this is called
+when we do Foo[int] - there, with cls=Foo and params=int.
+
However, note that this method is also called when defining generic
+classes in the first place with class Foo[T]: ….
Read-only property returning the kind of the test suite.
+
Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+language construct.
Read-only property returning the duration of a test entities run.
+
This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+distinguishable, assign setup and teardown durations with zero.
At least, parameterizing a generic class is the main thing this
+method does. For example, for some generic class Foo, this is called
+when we do Foo[int] - there, with cls=Foo and params=int.
+
However, note that this method is also called when defining generic
+classes in the first place with class Foo[T]: ….
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+Reader for JUnit unit testing summary files in XML format.
+"""
+from pathlib import Path
+from time import perf_counter_ns
+from typing import Optional as Nullable, Generator, Tuple, Union, TypeVar, Type, ClassVar
+
+from lxml.etree import ElementTree, Element, SubElement, tostring, _Element
+from pyTooling.Common import firstValue
+from pyTooling.Decorators import export, InheritDocString
+
+from pyEDAA.Reports.Unittesting import UnittestException, TestsuiteKind
+from pyEDAA.Reports.Unittesting import TestcaseStatus, TestsuiteStatus, IterationScheme
+from pyEDAA.Reports.Unittesting import TestsuiteSummary as ut_TestsuiteSummary, Testsuite as ut_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import Testcase as ju_Testcase, Testclass as ju_Testclass, Testsuite as ju_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import TestsuiteSummary as ju_TestsuiteSummary, Document as ju_Document
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int]
+
+
+@export
+@InheritDocString(ju_Testcase, merge=True)
+class Testcase(ju_Testcase):
+ """
+ This is a derived implementation for the Ant + JUnit4 dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testclass, merge=True)
+class Testclass(ju_Testclass):
+ """
+ This is a derived implementation for the Ant + JUnit4 dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testsuite, merge=True)
+class Testsuite(ju_Testsuite):
+ """
+ This is a derived implementation for the Ant + JUnit4 dialect.
+ """
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ for testclass in self._testclasses.values(): # type: Testclass
+ _ = testclass.Aggregate(strict)
+
+ tests += 1
+
+ status = testclass._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with state 'Unknown'.")
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testclass '{testclass._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ @classmethod
+ def FromTestsuite(cls, testsuite: ut_Testsuite) -> "Testsuite":
+ """
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object
+ adhering to the Ant + JUnit4 dialect.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model (Ant + JUnit4 dialect).
+ """
+ juTestsuite = cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status= testsuite._status,
+ )
+
+ juTestsuite._tests = testsuite._tests
+ juTestsuite._skipped = testsuite._skipped
+ juTestsuite._errored = testsuite._errored
+ juTestsuite._failed = testsuite._failed
+ juTestsuite._passed = testsuite._passed
+
+ for tc in testsuite.IterateTestcases():
+ ts = tc._parent
+ if ts is None:
+ raise UnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname = ts._name
+ ts = ts._parent
+ while ts is not None and ts._kind > TestsuiteKind.Logical:
+ classname = f"{ts._name}.{classname}"
+ ts = ts._parent
+
+ if classname in juTestsuite._testclasses:
+ juClass = juTestsuite._testclasses[classname]
+ else:
+ juClass = Testclass(classname, parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ return juTestsuite
+
+
+@export
+@InheritDocString(ju_TestsuiteSummary, merge=True)
+class TestsuiteSummary(ju_TestsuiteSummary):
+ """
+ This is a derived implementation for the Ant + JUnit4 dialect.
+ """
+
+ @classmethod
+ def FromTestsuiteSummary(cls, testsuiteSummary: ut_TestsuiteSummary) -> "TestsuiteSummary":
+ """
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite
+ summary object adhering to the Ant + JUnit4 dialect.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model (Ant + JUnit4 dialect).
+ """
+ return cls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+ )
+
+
+@export
+class Document(ju_Document):
+ """
+ A document reader and writer for the Ant + JUnit4 XML file format.
+
+ This class reads, validates and transforms an XML file in the Ant + JUnit4 format into a JUnit data model. It can then
+ be converted into a unified test entity data model.
+
+ In reverse, a JUnit data model instance with the specific Ant + JUnit4 file format can be created from a unified test
+ entity data model. This data model can be written as XML into a file.
+ """
+
+ _TESTCASE: ClassVar[Type[Testcase]] = Testcase
+ _TESTCLASS: ClassVar[Type[Testclass]] = Testclass
+ _TESTSUITE: ClassVar[Type[Testsuite]] = Testsuite
+
+ @classmethod
+ def FromTestsuiteSummary(cls, xmlReportFile: Path, testsuiteSummary: ut_TestsuiteSummary):
+ doc = cls(xmlReportFile)
+ doc._name = testsuiteSummary._name
+ doc._startTime = testsuiteSummary._startTime
+ doc._duration = testsuiteSummary._totalDuration
+ doc._status = testsuiteSummary._status
+ doc._tests = testsuiteSummary._tests
+ doc._skipped = testsuiteSummary._skipped
+ doc._errored = testsuiteSummary._errored
+ doc._failed = testsuiteSummary._failed
+ doc._passed = testsuiteSummary._passed
+
+ doc.AddTestsuites(Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+
+ return doc
+
+ def Analyze(self) -> None:
+ """
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`..
+
+ The used XML schema definition is specific to the Ant JUnit4 dialect.
+ """
+ xmlSchemaFile = "Ant-JUnit4.xsd"
+ self._Analyze(xmlSchemaFile)
+
+ def Write(self, path: Nullable[Path] = None, overwrite: bool = False, regenerate: bool = False) -> None:
+ """
+ Write the data model as XML into a file adhering to the Ant + JUnit4 dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ if path is None:
+ path = self._path
+
+ if not overwrite and path.exists():
+ raise UnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ from FileExistsError(f"File '{path}' already exists.")
+
+ if regenerate:
+ self.Generate(overwrite=True)
+
+ if self._xmlDocument is None:
+ ex = UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raise ex
+
+ try:
+ with path.open("wb") as file:
+ file.write(tostring(self._xmlDocument, encoding="utf-8", xml_declaration=True, pretty_print=True))
+ except Exception as ex:
+ raise UnittestException(f"JUnit XML file '{path}' can not be written.") from ex
+
+ def Convert(self) -> None:
+ """
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ if self._xmlDocument is None:
+ ex = UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raise ex
+
+ startConversion = perf_counter_ns()
+ rootElement: _Element = self._xmlDocument.getroot()
+
+ self._name = self._ConvertName(rootElement, optional=True)
+ self._startTime =self._ConvertTimestamp(rootElement, optional=True)
+ self._duration = self._ConvertTime(rootElement, optional=True)
+
+ # tests = rootElement.getAttribute("tests")
+ # skipped = rootElement.getAttribute("skipped")
+ # errors = rootElement.getAttribute("errors")
+ # failures = rootElement.getAttribute("failures")
+ # assertions = rootElement.getAttribute("assertions")
+
+ ts = Testsuite(self._name, startTime=self._startTime, duration=self._duration, parent=self)
+ self._ConvertTestsuiteChildren(rootElement, ts)
+
+ self.Aggregate()
+ endConversation = perf_counter_ns()
+ self._modelConversion = (endConversation - startConversion) / 1e9
+
+ def _ConvertTestsuite(self, parent: TestsuiteSummary, testsuitesNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite = Testsuite(
+ self._ConvertName(testsuitesNode, optional=False),
+ self._ConvertHostname(testsuitesNode, optional=False),
+ self._ConvertTimestamp(testsuitesNode, optional=False),
+ self._ConvertTime(testsuitesNode, optional=False),
+ parent=parent
+ )
+
+ self._ConvertTestsuiteChildren(testsuitesNode, newTestsuite)
+
+ def Generate(self, overwrite: bool = False) -> None:
+ """
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ if not overwrite and self._xmlDocument is not None:
+ raise UnittestException(f"Internal XML document is populated with data.")
+
+ if self.TestsuiteCount != 1:
+ ex = UnittestException(f"The Ant + JUnit4 format requires exactly one test suite.")
+ ex.add_note(f"Found {self.TestsuiteCount} test suites.")
+ raise ex
+
+ testsuite = firstValue(self._testsuites)
+
+ rootElement = Element("testsuite")
+ rootElement.attrib["name"] = self._name
+ if self._startTime is not None:
+ rootElement.attrib["timestamp"] = f"{self._startTime.isoformat()}"
+ if self._duration is not None:
+ rootElement.attrib["time"] = f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"] = str(self._tests)
+ rootElement.attrib["failures"] = str(self._failed)
+ rootElement.attrib["errors"] = str(self._errored)
+ rootElement.attrib["skipped"] = str(self._skipped)
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+ if testsuite._hostname is not None:
+ rootElement.attrib["hostname"] = testsuite._hostname
+
+ self._xmlDocument = ElementTree(rootElement)
+
+ for testclass in testsuite._testclasses.values():
+ for tc in testclass._testcases.values():
+ self._GenerateTestcase(tc, rootElement)
+
+ def _GenerateTestcase(self, testcase: Testcase, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement = SubElement(parentElement, "testcase")
+ if testcase.Classname is not None:
+ testcaseElement.attrib["classname"] = testcase.Classname
+ testcaseElement.attrib["name"] = testcase._name
+ if testcase._duration is not None:
+ testcaseElement.attrib["time"] = f"{testcase._duration.total_seconds():.6f}"
+ if testcase._assertionCount is not None:
+ testcaseElement.attrib["assertions"] = f"{testcase._assertionCount}"
+
+ if testcase._status is TestcaseStatus.Passed:
+ pass
+ elif testcase._status is TestcaseStatus.Failed:
+ failureElement = SubElement(testcaseElement, "failure")
+ elif testcase._status is TestcaseStatus.Skipped:
+ skippedElement = SubElement(testcaseElement, "skipped")
+ else:
+ errorElement = SubElement(testcaseElement, "error")
+
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+Reader for JUnit unit testing summary files in XML format.
+"""
+from pathlib import Path
+from time import perf_counter_ns
+from typing import Optional as Nullable, Generator, Tuple, Union, TypeVar, Type, ClassVar
+
+from lxml.etree import ElementTree, Element, SubElement, tostring, _Element
+from pyTooling.Common import firstValue
+from pyTooling.Decorators import export, InheritDocString
+
+from pyEDAA.Reports.Unittesting import UnittestException, TestsuiteKind
+from pyEDAA.Reports.Unittesting import TestcaseStatus, TestsuiteStatus, IterationScheme
+from pyEDAA.Reports.Unittesting import TestsuiteSummary as ut_TestsuiteSummary, Testsuite as ut_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import Testcase as ju_Testcase, Testclass as ju_Testclass, Testsuite as ju_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import TestsuiteSummary as ju_TestsuiteSummary, Document as ju_Document
+
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int]
+
+
+@export
+@InheritDocString(ju_Testcase, merge=True)
+class Testcase(ju_Testcase):
+ """
+ This is a derived implementation for the CTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testclass, merge=True)
+class Testclass(ju_Testclass):
+ """
+ This is a derived implementation for the CTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testsuite, merge=True)
+class Testsuite(ju_Testsuite):
+ """
+ This is a derived implementation for the CTest JUnit dialect.
+ """
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ for testclass in self._testclasses.values():
+ _ = testclass.Aggregate(strict)
+
+ tests += 1
+
+ status = testclass._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with state 'Unknown'.")
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testclass '{testclass._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ @classmethod
+ def FromTestsuite(cls, testsuite: ut_Testsuite) -> "Testsuite":
+ """
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object
+ adhering to the CTest JUnit dialect.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model (CTest JUnit dialect).
+ """
+ juTestsuite = cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status= testsuite._status,
+ )
+
+ juTestsuite._tests = testsuite._tests
+ juTestsuite._skipped = testsuite._skipped
+ juTestsuite._errored = testsuite._errored
+ juTestsuite._failed = testsuite._failed
+ juTestsuite._passed = testsuite._passed
+
+ for tc in testsuite.IterateTestcases():
+ ts = tc._parent
+ if ts is None:
+ raise UnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname = ts._name
+ ts = ts._parent
+ while ts is not None and ts._kind > TestsuiteKind.Logical:
+ classname = f"{ts._name}.{classname}"
+ ts = ts._parent
+
+ if classname in juTestsuite._testclasses:
+ juClass = juTestsuite._testclasses[classname]
+ else:
+ juClass = Testclass(classname, parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ return juTestsuite
+
+
+@export
+@InheritDocString(ju_TestsuiteSummary, merge=True)
+class TestsuiteSummary(ju_TestsuiteSummary):
+ """
+ This is a derived implementation for the CTest JUnit dialect.
+ """
+
+ @classmethod
+ def FromTestsuiteSummary(cls, testsuiteSummary: ut_TestsuiteSummary) -> "TestsuiteSummary":
+ """
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite
+ summary object adhering to the CTest JUnit dialect.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model (CTest JUnit dialect).
+ """
+ return cls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+ )
+
+
+@export
+class Document(ju_Document):
+ """
+ A document reader and writer for the CTest JUnit XML file format.
+
+ This class reads, validates and transforms an XML file in the CTest JUnit format into a JUnit data model. It can then
+ be converted into a unified test entity data model.
+
+ In reverse, a JUnit data model instance with the specific CTest JUnit file format can be created from a unified test
+ entity data model. This data model can be written as XML into a file.
+ """
+
+ _TESTCASE: ClassVar[Type[Testcase]] = Testcase
+ _TESTCLASS: ClassVar[Type[Testclass]] = Testclass
+ _TESTSUITE: ClassVar[Type[Testsuite]] = Testsuite
+
+ @classmethod
+ def FromTestsuiteSummary(cls, xmlReportFile: Path, testsuiteSummary: ut_TestsuiteSummary):
+ doc = cls(xmlReportFile)
+ doc._name = testsuiteSummary._name
+ doc._startTime = testsuiteSummary._startTime
+ doc._duration = testsuiteSummary._totalDuration
+ doc._status = testsuiteSummary._status
+ doc._tests = testsuiteSummary._tests
+ doc._skipped = testsuiteSummary._skipped
+ doc._errored = testsuiteSummary._errored
+ doc._failed = testsuiteSummary._failed
+ doc._passed = testsuiteSummary._passed
+
+ doc.AddTestsuites(Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+
+ return doc
+
+ def Analyze(self) -> None:
+ """
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`.
+
+ The used XML schema definition is specific to the CTest JUnit dialect.
+ """
+ xmlSchemaFile = "CTest-JUnit.xsd"
+ self._Analyze(xmlSchemaFile)
+
+ def Write(self, path: Nullable[Path] = None, overwrite: bool = False, regenerate: bool = False) -> None:
+ """
+ Write the data model as XML into a file adhering to the CTest dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ if path is None:
+ path = self._path
+
+ if not overwrite and path.exists():
+ raise UnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ from FileExistsError(f"File '{path}' already exists.")
+
+ if regenerate:
+ self.Generate(overwrite=True)
+
+ if self._xmlDocument is None:
+ ex = UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raise ex
+
+ try:
+ with path.open("wb") as file:
+ file.write(tostring(self._xmlDocument, encoding="utf-8", xml_declaration=True, pretty_print=True))
+ except Exception as ex:
+ raise UnittestException(f"JUnit XML file '{path}' can not be written.") from ex
+
+ def Convert(self) -> None:
+ """
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ if self._xmlDocument is None:
+ ex = UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raise ex
+
+ startConversion = perf_counter_ns()
+ rootElement: _Element = self._xmlDocument.getroot()
+
+ self._name = self._ConvertName(rootElement, optional=True)
+ self._startTime =self._ConvertTimestamp(rootElement, optional=True)
+ self._duration = self._ConvertTime(rootElement, optional=True)
+
+ # tests = rootElement.getAttribute("tests")
+ # skipped = rootElement.getAttribute("skipped")
+ # errors = rootElement.getAttribute("errors")
+ # failures = rootElement.getAttribute("failures")
+ # assertions = rootElement.getAttribute("assertions")
+
+ ts = Testsuite(self._name, startTime=self._startTime, duration=self._duration, parent=self)
+ self._ConvertTestsuiteChildren(rootElement, ts)
+
+ self.Aggregate()
+ endConversation = perf_counter_ns()
+ self._modelConversion = (endConversation - startConversion) / 1e9
+
+ def _ConvertTestsuite(self, parent: TestsuiteSummary, testsuitesNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite = Testsuite(
+ self._ConvertName(testsuitesNode, optional=False),
+ self._ConvertHostname(testsuitesNode, optional=False),
+ self._ConvertTimestamp(testsuitesNode, optional=False),
+ self._ConvertTime(testsuitesNode, optional=False),
+ parent=parent
+ )
+
+ self._ConvertTestsuiteChildren(testsuitesNode, newTestsuite)
+
+ def Generate(self, overwrite: bool = False) -> None:
+ """
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ if not overwrite and self._xmlDocument is not None:
+ raise UnittestException(f"Internal XML document is populated with data.")
+
+ if self.TestsuiteCount != 1:
+ ex = UnittestException(f"The CTest JUnit format requires exactly one test suite.")
+ ex.add_note(f"Found {self.TestsuiteCount} test suites.")
+ raise ex
+
+ testsuite = firstValue(self._testsuites)
+
+ rootElement = Element("testsuite")
+ rootElement.attrib["name"] = self._name
+ if self._startTime is not None:
+ rootElement.attrib["timestamp"] = f"{self._startTime.isoformat()}"
+ if self._duration is not None:
+ rootElement.attrib["time"] = f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"] = str(self._tests)
+ rootElement.attrib["failures"] = str(self._failed)
+ # rootElement.attrib["errors"] = str(self._errored)
+ rootElement.attrib["skipped"] = str(self._skipped)
+ rootElement.attrib["disabled"] = "0" # TODO: find a value
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+ rootElement.attrib["hostname"] = str(testsuite._hostname) # TODO: find a value
+
+ self._xmlDocument = ElementTree(rootElement)
+
+ for testclass in testsuite._testclasses.values():
+ for tc in testclass._testcases.values():
+ self._GenerateTestcase(tc, rootElement)
+
+ def _GenerateTestcase(self, testcase: Testcase, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement = SubElement(parentElement, "testcase")
+ if testcase.Classname is not None:
+ testcaseElement.attrib["classname"] = testcase.Classname
+ testcaseElement.attrib["name"] = testcase._name
+ if testcase._duration is not None:
+ testcaseElement.attrib["time"] = f"{testcase._duration.total_seconds():.6f}"
+ if testcase._assertionCount is not None:
+ testcaseElement.attrib["assertions"] = f"{testcase._assertionCount}"
+
+ testcaseElement.attrib["status"] = "run" # TODO: find a value
+
+ if testcase._status is TestcaseStatus.Passed:
+ pass
+ elif testcase._status is TestcaseStatus.Failed:
+ failureElement = SubElement(testcaseElement, "failure")
+ elif testcase._status is TestcaseStatus.Skipped:
+ skippedElement = SubElement(testcaseElement, "skipped")
+ else:
+ errorElement = SubElement(testcaseElement, "error")
+
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+Reader for JUnit unit testing summary files in XML format.
+"""
+from pathlib import Path
+from time import perf_counter_ns
+from typing import Optional as Nullable, Generator, Tuple, Union, TypeVar, Type, ClassVar
+
+from lxml.etree import ElementTree, Element, SubElement, tostring, _Element
+from pyTooling.Decorators import export, InheritDocString
+
+from pyEDAA.Reports.Unittesting import UnittestException, TestsuiteKind
+from pyEDAA.Reports.Unittesting import TestcaseStatus, TestsuiteStatus, IterationScheme
+from pyEDAA.Reports.Unittesting import TestsuiteSummary as ut_TestsuiteSummary, Testsuite as ut_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import Testcase as ju_Testcase, Testclass as ju_Testclass, Testsuite as ju_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import TestsuiteSummary as ju_TestsuiteSummary, Document as ju_Document
+
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int]
+
+
+@export
+@InheritDocString(ju_Testcase, merge=True)
+class Testcase(ju_Testcase):
+ """
+ This is a derived implementation for the GoogleTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testclass, merge=True)
+class Testclass(ju_Testclass):
+ """
+ This is a derived implementation for the GoogleTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testsuite, merge=True)
+class Testsuite(ju_Testsuite):
+ """
+ This is a derived implementation for the GoogleTest JUnit dialect.
+ """
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ for testclass in self._testclasses.values():
+ _ = testclass.Aggregate(strict)
+
+ tests += 1
+
+ status = testclass._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with state 'Unknown'.")
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testclass '{testclass._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ @classmethod
+ def FromTestsuite(cls, testsuite: ut_Testsuite) -> "Testsuite":
+ """
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object
+ adhering to the GoogleTest JUnit dialect.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model (GoogleTest JUnit dialect).
+ """
+ juTestsuite = cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status= testsuite._status,
+ )
+
+ juTestsuite._tests = testsuite._tests
+ juTestsuite._skipped = testsuite._skipped
+ juTestsuite._errored = testsuite._errored
+ juTestsuite._failed = testsuite._failed
+ juTestsuite._passed = testsuite._passed
+
+ for tc in testsuite.IterateTestcases():
+ ts = tc._parent
+ if ts is None:
+ raise UnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname = ts._name
+ ts = ts._parent
+ while ts is not None and ts._kind > TestsuiteKind.Logical:
+ classname = f"{ts._name}.{classname}"
+ ts = ts._parent
+
+ if classname in juTestsuite._testclasses:
+ juClass = juTestsuite._testclasses[classname]
+ else:
+ juClass = Testclass(classname, parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ return juTestsuite
+
+
+@export
+@InheritDocString(ju_TestsuiteSummary, merge=True)
+class TestsuiteSummary(ju_TestsuiteSummary):
+ """
+ This is a derived implementation for the GoogleTest JUnit dialect.
+ """
+
+ @classmethod
+ def FromTestsuiteSummary(cls, testsuiteSummary: ut_TestsuiteSummary) -> "TestsuiteSummary":
+ """
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite
+ summary object adhering to the GoogleTest JUnit dialect.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model (GoogleTest JUnit dialect).
+ """
+ return cls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+ )
+
+
+@export
+class Document(ju_Document):
+ """
+ A document reader and writer for the GoogelTest JUnit XML file format.
+
+ This class reads, validates and transforms an XML file in the GoogelTest JUnit format into a JUnit data model. It can
+ then be converted into a unified test entity data model.
+
+ In reverse, a JUnit data model instance with the specific GoogelTest JUnit file format can be created from a unified
+ test entity data model. This data model can be written as XML into a file.
+ """
+
+ _TESTCASE: ClassVar[Type[Testcase]] = Testcase
+ _TESTCLASS: ClassVar[Type[Testclass]] = Testclass
+ _TESTSUITE: ClassVar[Type[Testsuite]] = Testsuite
+
+ @classmethod
+ def FromTestsuiteSummary(cls, xmlReportFile: Path, testsuiteSummary: ut_TestsuiteSummary):
+ doc = cls(xmlReportFile)
+ doc._name = testsuiteSummary._name
+ doc._startTime = testsuiteSummary._startTime
+ doc._duration = testsuiteSummary._totalDuration
+ doc._status = testsuiteSummary._status
+ doc._tests = testsuiteSummary._tests
+ doc._skipped = testsuiteSummary._skipped
+ doc._errored = testsuiteSummary._errored
+ doc._failed = testsuiteSummary._failed
+ doc._passed = testsuiteSummary._passed
+
+ doc.AddTestsuites(Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+
+ return doc
+
+ def Analyze(self) -> None:
+ """
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`.
+
+ The used XML schema definition is specific to the GoogleTest JUnit dialect.
+ """
+ xmlSchemaFile = "GoogleTest-JUnit.xsd"
+ self._Analyze(xmlSchemaFile)
+
+ def Write(self, path: Nullable[Path] = None, overwrite: bool = False, regenerate: bool = False) -> None:
+ """
+ Write the data model as XML into a file adhering to the GoogleTest dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ if path is None:
+ path = self._path
+
+ if not overwrite and path.exists():
+ raise UnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ from FileExistsError(f"File '{path}' already exists.")
+
+ if regenerate:
+ self.Generate(overwrite=True)
+
+ if self._xmlDocument is None:
+ ex = UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raise ex
+
+ try:
+ with path.open("wb") as file:
+ file.write(tostring(self._xmlDocument, encoding="utf-8", xml_declaration=True, pretty_print=True))
+ except Exception as ex:
+ raise UnittestException(f"JUnit XML file '{path}' can not be written.") from ex
+
+ def Convert(self) -> None:
+ """
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ if self._xmlDocument is None:
+ ex = UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raise ex
+
+ startConversion = perf_counter_ns()
+ rootElement: _Element = self._xmlDocument.getroot()
+
+ self._name = self._ConvertName(rootElement, optional=True)
+ self._startTime =self._ConvertTimestamp(rootElement, optional=True)
+ self._duration = self._ConvertTime(rootElement, optional=True)
+
+ # tests = rootElement.getAttribute("tests")
+ # skipped = rootElement.getAttribute("skipped")
+ # errors = rootElement.getAttribute("errors")
+ # failures = rootElement.getAttribute("failures")
+ # assertions = rootElement.getAttribute("assertions")
+
+ for rootNode in rootElement.iterchildren(tag="testsuite"): # type: _Element
+ self._ConvertTestsuite(self, rootNode)
+
+ self.Aggregate()
+ endConversation = perf_counter_ns()
+ self._modelConversion = (endConversation - startConversion) / 1e9
+
+ def _ConvertTestsuite(self, parent: TestsuiteSummary, testsuitesNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite = Testsuite(
+ self._ConvertName(testsuitesNode, optional=False),
+ self._ConvertHostname(testsuitesNode, optional=True),
+ self._ConvertTimestamp(testsuitesNode, optional=False),
+ self._ConvertTime(testsuitesNode, optional=False),
+ parent=parent
+ )
+
+ self._ConvertTestsuiteChildren(testsuitesNode, newTestsuite)
+
+ def Generate(self, overwrite: bool = False) -> None:
+ """
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuites>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ if not overwrite and self._xmlDocument is not None:
+ raise UnittestException(f"Internal XML document is populated with data.")
+
+ rootElement = Element("testsuites")
+ rootElement.attrib["name"] = self._name
+ if self._startTime is not None:
+ rootElement.attrib["timestamp"] = f"{self._startTime.isoformat()}"
+ if self._duration is not None:
+ rootElement.attrib["time"] = f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"] = str(self._tests)
+ rootElement.attrib["failures"] = str(self._failed)
+ rootElement.attrib["errors"] = str(self._errored)
+ # rootElement.attrib["skipped"] = str(self._skipped)
+ rootElement.attrib["disabled"] = "0" # TODO: find a value
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+
+ self._xmlDocument = ElementTree(rootElement)
+
+ for testsuite in self._testsuites.values():
+ self._GenerateTestsuite(testsuite, rootElement)
+
+ def _GenerateTestsuite(self, testsuite: Testsuite, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test suite.
+
+ This method generates the XML element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param testsuite: The test suite to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testsuiteElement = SubElement(parentElement, "testsuite")
+ testsuiteElement.attrib["name"] = testsuite._name
+ if testsuite._startTime is not None:
+ testsuiteElement.attrib["timestamp"] = f"{testsuite._startTime.isoformat()}"
+ if testsuite._duration is not None:
+ testsuiteElement.attrib["time"] = f"{testsuite._duration.total_seconds():.6f}"
+ testsuiteElement.attrib["tests"] = str(testsuite._tests)
+ testsuiteElement.attrib["failures"] = str(testsuite._failed)
+ testsuiteElement.attrib["errors"] = str(testsuite._errored)
+ testsuiteElement.attrib["skipped"] = str(testsuite._skipped)
+ testsuiteElement.attrib["disabled"] = "0" # TODO: find a value
+ # if testsuite._assertionCount is not None:
+ # testsuiteElement.attrib["assertions"] = f"{testsuite._assertionCount}"
+ # if testsuite._hostname is not None:
+ # testsuiteElement.attrib["hostname"] = testsuite._hostname
+
+ for testclass in testsuite._testclasses.values():
+ for tc in testclass._testcases.values():
+ self._GenerateTestcase(tc, testsuiteElement)
+
+ def _GenerateTestcase(self, testcase: Testcase, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement = SubElement(parentElement, "testcase")
+ if testcase.Classname is not None:
+ testcaseElement.attrib["classname"] = testcase.Classname
+ testcaseElement.attrib["name"] = testcase._name
+ if testcase._duration is not None:
+ testcaseElement.attrib["time"] = f"{testcase._duration.total_seconds():.6f}"
+ if testcase._assertionCount is not None:
+ testcaseElement.attrib["assertions"] = f"{testcase._assertionCount}"
+
+ testcaseElement.attrib["timestamp"] = f"{testcase._parent._parent._startTime.isoformat()}" # TODO: find a value
+ testcaseElement.attrib["file"] = "" # TODO: find a value
+ testcaseElement.attrib["line"] = "0" # TODO: find a value
+ testcaseElement.attrib["status"] = "run" # TODO: find a value
+ testcaseElement.attrib["result"] = "completed" # TODO: find a value
+
+ if testcase._status is TestcaseStatus.Passed:
+ pass
+ elif testcase._status is TestcaseStatus.Failed:
+ failureElement = SubElement(testcaseElement, "failure")
+ elif testcase._status is TestcaseStatus.Skipped:
+ skippedElement = SubElement(testcaseElement, "skipped")
+ else:
+ errorElement = SubElement(testcaseElement, "error")
+
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+Reader for JUnit unit testing summary files in XML format.
+"""
+from pathlib import Path
+from time import perf_counter_ns
+from typing import Optional as Nullable, Generator, Tuple, Union, TypeVar, Type, ClassVar
+
+from lxml.etree import ElementTree, Element, SubElement, tostring, _Element
+from pyTooling.Decorators import export, InheritDocString
+
+from pyEDAA.Reports.Unittesting import UnittestException, TestsuiteKind
+from pyEDAA.Reports.Unittesting import TestcaseStatus, TestsuiteStatus, IterationScheme
+from pyEDAA.Reports.Unittesting import TestsuiteSummary as ut_TestsuiteSummary, Testsuite as ut_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import Testcase as ju_Testcase, Testclass as ju_Testclass, Testsuite as ju_Testsuite
+from pyEDAA.Reports.Unittesting.JUnit import TestsuiteSummary as ju_TestsuiteSummary, Document as ju_Document
+
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int]
+
+
+@export
+@InheritDocString(ju_Testcase, merge=True)
+class Testcase(ju_Testcase):
+ """
+ This is a derived implementation for the pyTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testclass, merge=True)
+class Testclass(ju_Testclass):
+ """
+ This is a derived implementation for the pyTest JUnit dialect.
+ """
+
+
+@export
+@InheritDocString(ju_Testsuite, merge=True)
+class Testsuite(ju_Testsuite):
+ """
+ This is a derived implementation for the pyTest JUnit dialect.
+ """
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ for testclass in self._testclasses.values():
+ _ = testclass.Aggregate(strict)
+
+ tests += 1
+
+ status = testclass._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with state 'Unknown'.")
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testclass '{testclass._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ @classmethod
+ def FromTestsuite(cls, testsuite: ut_Testsuite) -> "Testsuite":
+ """
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object
+ adhering to the pyTest JUnit dialect.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model (pyTest JUnitdialect).
+ """
+ juTestsuite = cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status= testsuite._status,
+ )
+
+ juTestsuite._tests = testsuite._tests
+ juTestsuite._skipped = testsuite._skipped
+ juTestsuite._errored = testsuite._errored
+ juTestsuite._failed = testsuite._failed
+ juTestsuite._passed = testsuite._passed
+
+ for tc in testsuite.IterateTestcases():
+ ts = tc._parent
+ if ts is None:
+ raise UnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname = ts._name
+ ts = ts._parent
+ while ts is not None and ts._kind > TestsuiteKind.Logical:
+ classname = f"{ts._name}.{classname}"
+ ts = ts._parent
+
+ if classname in juTestsuite._testclasses:
+ juClass = juTestsuite._testclasses[classname]
+ else:
+ juClass = Testclass(classname, parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ return juTestsuite
+
+
+@export
+@InheritDocString(ju_TestsuiteSummary, merge=True)
+class TestsuiteSummary(ju_TestsuiteSummary):
+ """
+ This is a derived implementation for the pyTest JUnit dialect.
+ """
+
+ @classmethod
+ def FromTestsuiteSummary(cls, testsuiteSummary: ut_TestsuiteSummary) -> "TestsuiteSummary":
+ """
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite
+ summary object adhering to the pyTest JUnit dialect.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model (pyTest JUnit dialect).
+ """
+ return cls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+ )
+
+
+@export
+class Document(ju_Document):
+ """
+ A document reader and writer for the pyTest JUnit XML file format.
+
+ This class reads, validates and transforms an XML file in the pyTest JUnit format into a JUnit data model. It can then
+ be converted into a unified test entity data model.
+
+ In reverse, a JUnit data model instance with the specific pyTest JUnit file format can be created from a unified test
+ entity data model. This data model can be written as XML into a file.
+ """
+
+ _TESTCASE: ClassVar[Type[Testcase]] = Testcase
+ _TESTCLASS: ClassVar[Type[Testclass]] = Testclass
+ _TESTSUITE: ClassVar[Type[Testsuite]] = Testsuite
+
+ @classmethod
+ def FromTestsuiteSummary(cls, xmlReportFile: Path, testsuiteSummary: ut_TestsuiteSummary):
+ doc = cls(xmlReportFile)
+ doc._name = testsuiteSummary._name
+ doc._startTime = testsuiteSummary._startTime
+ doc._duration = testsuiteSummary._totalDuration
+ doc._status = testsuiteSummary._status
+ doc._tests = testsuiteSummary._tests
+ doc._skipped = testsuiteSummary._skipped
+ doc._errored = testsuiteSummary._errored
+ doc._failed = testsuiteSummary._failed
+ doc._passed = testsuiteSummary._passed
+
+ doc.AddTestsuites(Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+
+ return doc
+
+ def Analyze(self) -> None:
+ """
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`.
+
+ The used XML schema definition is specific to the pyTest JUnit dialect.
+ """
+ xmlSchemaFile = "PyTest-JUnit.xsd"
+ self._Analyze(xmlSchemaFile)
+
+ def Write(self, path: Nullable[Path] = None, overwrite: bool = False, regenerate: bool = False) -> None:
+ """
+ Write the data model as XML into a file adhering to the pyTest dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ if path is None:
+ path = self._path
+
+ if not overwrite and path.exists():
+ raise UnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ from FileExistsError(f"File '{path}' already exists.")
+
+ if regenerate:
+ self.Generate(overwrite=True)
+
+ if self._xmlDocument is None:
+ ex = UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raise ex
+
+ try:
+ with path.open("wb") as file:
+ file.write(tostring(self._xmlDocument, encoding="utf-8", xml_declaration=True, pretty_print=True))
+ except Exception as ex:
+ raise UnittestException(f"JUnit XML file '{path}' can not be written.") from ex
+
+ def Convert(self) -> None:
+ """
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ if self._xmlDocument is None:
+ ex = UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raise ex
+
+ startConversion = perf_counter_ns()
+ rootElement: _Element = self._xmlDocument.getroot()
+
+ self._name = self._ConvertName(rootElement, optional=True)
+ self._startTime =self._ConvertTimestamp(rootElement, optional=True)
+ self._duration = self._ConvertTime(rootElement, optional=True)
+
+ # tests = rootElement.getAttribute("tests")
+ # skipped = rootElement.getAttribute("skipped")
+ # errors = rootElement.getAttribute("errors")
+ # failures = rootElement.getAttribute("failures")
+ # assertions = rootElement.getAttribute("assertions")
+
+ for rootNode in rootElement.iterchildren(tag="testsuite"): # type: _Element
+ self._ConvertTestsuite(self, rootNode)
+
+ self.Aggregate()
+ endConversation = perf_counter_ns()
+ self._modelConversion = (endConversation - startConversion) / 1e9
+
+ def _ConvertTestsuite(self, parent: TestsuiteSummary, testsuitesNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite = Testsuite(
+ self._ConvertName(testsuitesNode, optional=False),
+ self._ConvertHostname(testsuitesNode, optional=False),
+ self._ConvertTimestamp(testsuitesNode, optional=False),
+ self._ConvertTime(testsuitesNode, optional=False),
+ parent=parent
+ )
+
+ self._ConvertTestsuiteChildren(testsuitesNode, newTestsuite)
+
+ def Generate(self, overwrite: bool = False) -> None:
+ """
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuites>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ if not overwrite and self._xmlDocument is not None:
+ raise UnittestException(f"Internal XML document is populated with data.")
+
+ rootElement = Element("testsuites")
+ rootElement.attrib["name"] = self._name
+ if self._startTime is not None:
+ rootElement.attrib["timestamp"] = f"{self._startTime.isoformat()}"
+ if self._duration is not None:
+ rootElement.attrib["time"] = f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"] = str(self._tests)
+ rootElement.attrib["failures"] = str(self._failed)
+ rootElement.attrib["errors"] = str(self._errored)
+ rootElement.attrib["skipped"] = str(self._skipped)
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+
+ self._xmlDocument = ElementTree(rootElement)
+
+ for testsuite in self._testsuites.values():
+ self._GenerateTestsuite(testsuite, rootElement)
+
+ def _GenerateTestsuite(self, testsuite: Testsuite, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test suite.
+
+ This method generates the XML element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param testsuite: The test suite to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testsuiteElement = SubElement(parentElement, "testsuite")
+ testsuiteElement.attrib["name"] = testsuite._name
+ if testsuite._startTime is not None:
+ testsuiteElement.attrib["timestamp"] = f"{testsuite._startTime.isoformat()}"
+ if testsuite._duration is not None:
+ testsuiteElement.attrib["time"] = f"{testsuite._duration.total_seconds():.6f}"
+ testsuiteElement.attrib["tests"] = str(testsuite._tests)
+ testsuiteElement.attrib["failures"] = str(testsuite._failed)
+ testsuiteElement.attrib["errors"] = str(testsuite._errored)
+ testsuiteElement.attrib["skipped"] = str(testsuite._skipped)
+ # if testsuite._assertionCount is not None:
+ # testsuiteElement.attrib["assertions"] = f"{testsuite._assertionCount}"
+ if testsuite._hostname is not None:
+ testsuiteElement.attrib["hostname"] = testsuite._hostname
+
+ for testclass in testsuite._testclasses.values():
+ for tc in testclass._testcases.values():
+ self._GenerateTestcase(tc, testsuiteElement)
+
+ def _GenerateTestcase(self, testcase: Testcase, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement = SubElement(parentElement, "testcase")
+ if testcase.Classname is not None:
+ testcaseElement.attrib["classname"] = testcase.Classname
+ testcaseElement.attrib["name"] = testcase._name
+ if testcase._duration is not None:
+ testcaseElement.attrib["time"] = f"{testcase._duration.total_seconds():.6f}"
+ if testcase._assertionCount is not None:
+ testcaseElement.attrib["assertions"] = f"{testcase._assertionCount}"
+
+ if testcase._status is TestcaseStatus.Passed:
+ pass
+ elif testcase._status is TestcaseStatus.Failed:
+ failureElement = SubElement(testcaseElement, "failure")
+ elif testcase._status is TestcaseStatus.Skipped:
+ skippedElement = SubElement(testcaseElement, "skipped")
+ else:
+ errorElement = SubElement(testcaseElement, "error")
+
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# Copyright 2023-2023 Patrick Lehmann - Bötzingen, Germany #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+The pyEDAA.Reports.Unittesting.JUnit package implements a hierarchy of test entities for the JUnit unit testing summary
+file format (XML format). This test entity hierarchy is not derived from :class:`pyEDAA.Reports.Unittesting`, because it
+doesn't match the unified data model. Nonetheless, both data models can be converted to each other. In addition, derived
+data models are provided for the many dialects of that XML file format. See the list modules in this package for the
+implemented dialects.
+
+The test entity hierarchy consists of test cases, test classes, test suites and a test summary. Test cases are the leaf
+elements in the hierarchy and represent an individual test run. Next, test classes group test cases, because the
+original Ant + JUnit format groups test cases (Java methods) in a Java class. Next, test suites are used to group
+multiple test classes. Finally, the root element is a test summary. When such a summary is stored in a file format like
+Ant + JUnit4 XML, a file format specific document is derived from a summary class.
+
+**Data Model**
+
+.. mermaid::
+
+ graph TD;
+ doc[Document]
+ sum[Summary]
+ ts1[Testsuite]
+ ts11[Testsuite]
+ ts2[Testsuite]
+
+ tc111[Testclass]
+ tc112[Testclass]
+ tc23[Testclass]
+
+ tc1111[Testcase]
+ tc1112[Testcase]
+ tc1113[Testcase]
+ tc1121[Testcase]
+ tc1122[Testcase]
+ tc231[Testcase]
+ tc232[Testcase]
+ tc233[Testcase]
+
+ doc:::root -.-> sum:::summary
+ sum --> ts1:::suite
+ sum ---> ts2:::suite
+ ts1 --> ts11:::suite
+
+ ts11 --> tc111:::cls
+ ts11 --> tc112:::cls
+ ts2 --> tc23:::cls
+
+ tc111 --> tc1111:::case
+ tc111 --> tc1112:::case
+ tc111 --> tc1113:::case
+ tc112 --> tc1121:::case
+ tc112 --> tc1122:::case
+ tc23 --> tc231:::case
+ tc23 --> tc232:::case
+ tc23 --> tc233:::case
+
+ classDef root fill:#4dc3ff
+ classDef summary fill:#80d4ff
+ classDef suite fill:#b3e6ff
+ classDef cls fill:#ff9966
+ classDef case fill:#eeccff
+"""
+from datetime import datetime, timedelta
+from enum import Flag
+from pathlib import Path
+from sys import version_info
+from time import perf_counter_ns
+from typing import Optional as Nullable, Iterable, Dict, Any, Generator, Tuple, Union, TypeVar, Type, ClassVar
+
+from lxml.etree import XMLParser, parse, XMLSchema, ElementTree, Element, SubElement, tostring
+from lxml.etree import XMLSyntaxError, _ElementTree, _Element, _Comment, XMLSchemaParseError
+from pyTooling.Common import getFullyQualifiedName, getResourceFile
+from pyTooling.Decorators import export, readonly
+from pyTooling.Exceptions import ToolingException
+from pyTooling.MetaClasses import ExtendedType, mustoverride, abstractmethod
+from pyTooling.Tree import Node
+
+from pyEDAA.Reports import Resources
+from pyEDAA.Reports.Unittesting import UnittestException, AlreadyInHierarchyException, DuplicateTestsuiteException, DuplicateTestcaseException
+from pyEDAA.Reports.Unittesting import TestcaseStatus, TestsuiteStatus, TestsuiteKind, IterationScheme
+from pyEDAA.Reports.Unittesting import Document as ut_Document, TestsuiteSummary as ut_TestsuiteSummary
+from pyEDAA.Reports.Unittesting import Testsuite as ut_Testsuite, Testcase as ut_Testcase
+
+
+@export
+class JUnitException:
+ """An exception-mixin for JUnit format specific exceptions."""
+
+
+@export
+class UnittestException(UnittestException, JUnitException):
+ pass
+
+
+@export
+class AlreadyInHierarchyException(AlreadyInHierarchyException, JUnitException):
+ """
+ A unit test exception raised if the element is already part of a hierarchy.
+
+ This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+ hierarchy should occur only once in the hierarchy.
+
+ .. hint::
+
+ This is usually caused by a non-None parent reference.
+ """
+
+
+@export
+class DuplicateTestsuiteException(DuplicateTestsuiteException, JUnitException):
+ """
+ A unit test exception raised on duplicate test suites (by name).
+
+ This exception is raised, if a child test suite with same name already exist in the test suite.
+
+ .. hint::
+
+ Test suite names need to be unique per parent element (test suite or test summary).
+ """
+
+
+@export
+class DuplicateTestcaseException(DuplicateTestcaseException, JUnitException):
+ """
+ A unit test exception raised on duplicate test cases (by name).
+
+ This exception is raised, if a child test case with same name already exist in the test suite.
+
+ .. hint::
+
+ Test case names need to be unique per parent element (test suite).
+ """
+
+
+@export
+class JUnitReaderMode(Flag):
+ Default = 0 #: Default behavior
+ DecoupleTestsuiteHierarchyAndTestcaseClassName = 1 #: Undocumented
+
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int]
+
+
+@export
+class Base(metaclass=ExtendedType, slots=True):
+ """
+ Base-class for all test entities (test cases, test classes, test suites, ...).
+
+ It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+ hierarchy.
+
+ Every test entity has a name to identity it. It's also used in the parent's child element dictionaries to identify the
+ child. |br|
+ E.g. it's used as a test case name in the dictionary of test cases in a test class.
+ """
+
+ _parent: Nullable["Testsuite"]
+ _name: str
+
+ def __init__(self, name: str, parent: Nullable["Testsuite"] = None):
+ """
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param parent: Reference to the parent test entity.
+ :raises ValueError: If parameter 'name' is None.
+ :raises TypeError: If parameter 'name' is not a string.
+ :raises ValueError: If parameter 'name' is empty.
+ """
+ if name is None:
+ raise ValueError(f"Parameter 'name' is None.")
+ elif not isinstance(name, str):
+ ex = TypeError(f"Parameter 'name' is not of type 'str'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(name)}'.")
+ raise ex
+ elif name.strip() == "":
+ raise ValueError(f"Parameter 'name' is empty.")
+
+ self._parent = parent
+ self._name = name
+
+ @readonly
+ def Parent(self) -> Nullable["Testsuite"]:
+ """
+ Read-only property returning the reference to the parent test entity.
+
+ :return: Reference to the parent entity.
+ """
+ return self._parent
+
+ # QUESTION: allow Parent as setter?
+
+ @readonly
+ def Name(self) -> str:
+ """
+ Read-only property returning the test entity's name.
+
+ :return:
+ """
+ return self._name
+
+
+@export
+class BaseWithProperties(Base):
+ """
+ Base-class for all test entities supporting properties (test cases, test suites, ...).
+
+ Every test entity has fields for the test duration and number of executed assertions.
+
+ Every test entity offers an internal dictionary for properties.
+ """
+
+ _duration: Nullable[timedelta]
+ _assertionCount: Nullable[int]
+ _properties: Dict[str, Any]
+
+ def __init__(
+ self,
+ name: str,
+ duration: Nullable[timedelta] = None,
+ assertionCount: Nullable[int] = None,
+ parent: Nullable["Testsuite"] = None
+ ):
+ """
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param duration: Duration of the entity's execution.
+ :param assertionCount: Number of assertions within the test.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'duration' is not a timedelta.
+ :raises TypeError: If parameter 'assertionCount' is not an integer.
+ """
+ super().__init__(name, parent)
+
+ if duration is not None and not isinstance(duration, timedelta):
+ ex = TypeError(f"Parameter 'duration' is not of type 'timedelta'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(duration)}'.")
+ raise ex
+
+ if assertionCount is not None and not isinstance(assertionCount, int):
+ ex = TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(assertionCount)}'.")
+ raise ex
+
+ self._duration = duration
+ self._assertionCount = assertionCount
+
+ self._properties = {}
+
+ @readonly
+ def Duration(self) -> timedelta:
+ """
+ Read-only property returning the duration of a test entity run.
+
+ .. note::
+
+ The JUnit format doesn't distinguish setup, run and teardown durations.
+
+ :return: Duration of the entity's execution.
+ """
+ return self._duration
+
+ @readonly
+ @abstractmethod
+ def AssertionCount(self) -> int:
+ """
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ .. note::
+
+ The JUnit format doesn't distinguish passed and failed assertions.
+
+ :return: Number of assertions.
+ """
+
+ def __len__(self) -> int:
+ """
+ Returns the number of annotated properties.
+
+ Syntax: :pycode:`length = len(obj)`
+
+ :return: Number of annotated properties.
+ """
+ return len(self._properties)
+
+ def __getitem__(self, name: str) -> Any:
+ """
+ Access a property by name.
+
+ Syntax: :pycode:`value = obj[name]`
+
+ :param name: Name if the property.
+ :return: Value of the accessed property.
+ """
+ return self._properties[name]
+
+ def __setitem__(self, name: str, value: Any) -> None:
+ """
+ Set the value of a property by name.
+
+ If the property doesn't exist yet, it's created.
+
+ Syntax: :pycode:`obj[name] = value`
+
+ :param name: Name of the property.
+ :param value: Value of the property.
+ """
+ self._properties[name] = value
+
+ def __delitem__(self, name: str) -> None:
+ """
+ Delete a property by name.
+
+ Syntax: :pycode:`del obj[name]`
+
+ :param name: Name if the property.
+ """
+ del self._properties[name]
+
+ def __contains__(self, name: str) -> bool:
+ """
+ Returns True, if a property was annotated by this name.
+
+ Syntax: :pycode:`name in obj`
+
+ :param name: Name of the property.
+ :return: True, if the property was annotated.
+ """
+ return name in self._properties
+
+ def __iter__(self) -> Generator[Tuple[str, Any], None, None]:
+ """
+ Iterate all annotated properties.
+
+ Syntax: :pycode:`for name, value in obj:`
+
+ :return: A generator of property tuples (name, value).
+ """
+ yield from self._properties.items()
+
+
+@export
+class Testcase(BaseWithProperties):
+ """
+ A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
+ Test cases are grouped by test classes in the test entity hierarchy. These are again grouped by test suites. The root
+ of the hierarchy is a test summary.
+
+ Every test case has an overall status like unknown, skipped, failed or passed.
+ """
+
+ _status: TestcaseStatus
+
+ def __init__(
+ self,
+ name: str,
+ duration: Nullable[timedelta] = None,
+ status: TestcaseStatus = TestcaseStatus.Unknown,
+ assertionCount: Nullable[int] = None,
+ parent: Nullable["Testclass"] = None
+ ):
+ """
+ Initializes the fields of a test case.
+
+ :param name: Name of the test entity.
+ :param duration: Duration of the entity's execution.
+ :param status: Status of the test case.
+ :param assertionCount: Number of assertions within the test.
+ :param parent: Reference to the parent test class.
+ :raises TypeError: If parameter 'parent' is not a Testsuite.
+ :raises ValueError: If parameter 'assertionCount' is not consistent.
+ """
+ if parent is not None:
+ if not isinstance(parent, Testclass):
+ ex = TypeError(f"Parameter 'parent' is not of type 'Testclass'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testcases[name] = self
+
+ super().__init__(name, duration, assertionCount, parent)
+
+ if not isinstance(status, TestcaseStatus):
+ ex = TypeError(f"Parameter 'status' is not of type 'TestcaseStatus'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(status)}'.")
+ raise ex
+
+ self._status = status
+
+ @readonly
+ def Classname(self) -> str:
+ """
+ Read-only property returning the class name of the test case.
+
+ :return: The test case's class name.
+
+ .. note::
+
+ In the JUnit format, a test case is uniquely identified by a tuple of class name and test case name. This
+ structure has been decomposed by this data model into 2 leaf-levels in the test entity hierarchy. Thus, the class
+ name is represented by its own level and instances of test classes.
+ """
+ if self._parent is None:
+ raise UnittestException("Standalone Testcase instance is not linked to a Testclass.")
+ return self._parent._name
+
+ @readonly
+ def Status(self) -> TestcaseStatus:
+ """
+ Read-only property returning the status of the test case.
+
+ :return: The test case's status.
+ """
+ return self._status
+
+ @readonly
+ def AssertionCount(self) -> int:
+ """
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ .. note::
+
+ The JUnit format doesn't distinguish passed and failed assertions.
+
+ :return: Number of assertions.
+ """
+ if self._assertionCount is None:
+ return 0
+ return self._assertionCount
+
+ def Copy(self) -> "Testcase":
+ return self.__class__(
+ self._name,
+ self._duration,
+ self._status,
+ self._assertionCount
+ )
+
+ def Aggregate(self) -> None:
+ if self._status is TestcaseStatus.Unknown:
+ if self._assertionCount is None:
+ self._status = TestcaseStatus.Passed
+ elif self._assertionCount == 0:
+ self._status = TestcaseStatus.Weak
+ else:
+ self._status = TestcaseStatus.Failed
+
+ # TODO: check for setup errors
+ # TODO: check for teardown errors
+
+ @classmethod
+ def FromTestcase(cls, testcase: ut_Testcase) -> "Testcase":
+ """
+ Convert a test case of the unified test entity data model to the JUnit specific data model's test case object.
+
+ :param testcase: Test case from unified data model.
+ :return: Test case from JUnit specific data model.
+ """
+ return cls(
+ testcase._name,
+ duration=testcase._testDuration,
+ status= testcase._status,
+ assertionCount=testcase._assertionCount
+ )
+
+ def ToTestcase(self) -> ut_Testcase:
+ return ut_Testcase(
+ self._name,
+ testDuration=self._duration,
+ status=self._status,
+ assertionCount=self._assertionCount,
+ # TODO: as only assertions are recorded by JUnit files, all are marked as passed
+ passedAssertionCount=self._assertionCount
+ )
+
+ def ToTree(self) -> Node:
+ node = Node(value=self._name)
+ node["status"] = self._status
+ node["assertionCount"] = self._assertionCount
+ node["duration"] = self._duration
+
+ return node
+
+ def __str__(self) -> str:
+ moduleName = self.__module__.split(".")[-1]
+ className = self.__class__.__name__
+ return (
+ f"<{moduleName}{className} {self._name}: {self._status.name} - asserts:{self._assertionCount}>"
+ )
+
+
+@export
+class TestsuiteBase(BaseWithProperties):
+ """
+ Base-class for all test suites and for test summaries.
+
+ A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+ element in that hierarchy. While a test suite groups test classes, a test summary can only group test suites. Thus, a
+ test summary contains no test classes and test cases.
+ """
+
+ _startTime: Nullable[datetime]
+ _status: TestsuiteStatus
+
+ _tests: int
+ _skipped: int
+ _errored: int
+ _failed: int
+ _passed: int
+
+ def __init__(
+ self,
+ name: str,
+ startTime: Nullable[datetime] = None,
+ duration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ parent: Nullable["Testsuite"] = None
+ ):
+ """
+ Initializes the based-class fields of a test suite or test summary.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param duration: Duration of the entity's execution.
+ :param status: Overall status of the test entity.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ """
+ if parent is not None:
+ if not isinstance(parent, TestsuiteBase):
+ ex = TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testsuites[name] = self
+
+ super().__init__(name, duration, None, parent)
+
+ self._startTime = startTime
+ self._status = status
+ self._tests = 0
+ self._skipped = 0
+ self._errored = 0
+ self._failed = 0
+ self._passed = 0
+
+ @readonly
+ def StartTime(self) -> Nullable[datetime]:
+ return self._startTime
+
+ @readonly
+ def Status(self) -> TestsuiteStatus:
+ return self._status
+
+ @readonly
+ @mustoverride
+ def TestcaseCount(self) -> int:
+ pass
+
+ @readonly
+ def Tests(self) -> int:
+ return self.TestcaseCount
+
+ @readonly
+ def Skipped(self) -> int:
+ return self._skipped
+
+ @readonly
+ def Errored(self) -> int:
+ return self._errored
+
+ @readonly
+ def Failed(self) -> int:
+ return self._failed
+
+ @readonly
+ def Passed(self) -> int:
+ return self._passed
+
+ def Aggregate(self) -> TestsuiteAggregateReturnType:
+ tests = 0
+ skipped = 0
+ errored = 0
+ failed = 0
+ passed = 0
+
+ # for testsuite in self._testsuites.values():
+ # t, s, e, w, f, p = testsuite.Aggregate()
+ # tests += t
+ # skipped += s
+ # errored += e
+ # weak += w
+ # failed += f
+ # passed += p
+
+ return tests, skipped, errored, failed, passed
+
+ @mustoverride
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[TestsuiteType, Testcase], None, None]:
+ pass
+
+
+@export
+class Testclass(Base):
+ """
+ A test class is a low-level element in the test entity hierarchy representing a group of tests.
+
+ Test classes contain test cases and are grouped by a test suites.
+ """
+
+ _testcases: Dict[str, "Testcase"]
+
+ def __init__(
+ self,
+ classname: str,
+ testcases: Nullable[Iterable["Testcase"]] = None,
+ parent: Nullable["Testsuite"] = None
+ ):
+ """
+ Initializes the fields of the test class.
+
+ :param classname: Classname of the test entity.
+ :param parent: Reference to the parent test suite.
+ :raises ValueError: If parameter 'classname' is None.
+ :raises TypeError: If parameter 'classname' is not a string.
+ :raises ValueError: If parameter 'classname' is empty.
+ """
+ if parent is not None:
+ if not isinstance(parent, Testsuite):
+ ex = TypeError(f"Parameter 'parent' is not of type 'Testsuite'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testclasses[classname] = self
+
+ super().__init__(classname, parent)
+
+ self._testcases = {}
+ if testcases is not None:
+ for testcase in testcases:
+ if testcase._parent is not None:
+ raise AlreadyInHierarchyException(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ if testcase._name in self._testcases:
+ raise DuplicateTestcaseException(f"Class already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent = self
+ self._testcases[testcase._name] = testcase
+
+ @readonly
+ def Classname(self) -> str:
+ """
+ Read-only property returning the name of the test class.
+
+ :return: The test class' name.
+ """
+ return self._name
+
+ @readonly
+ def Testcases(self) -> Dict[str, "Testcase"]:
+ """
+ Read-only property returning a reference to the internal dictionary of test cases.
+
+ :return: Reference to the dictionary of test cases.
+ """
+ return self._testcases
+
+ @readonly
+ def TestcaseCount(self) -> int:
+ """
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ return len(self._testcases)
+
+ @readonly
+ def AssertionCount(self) -> int:
+ return sum(tc.AssertionCount for tc in self._testcases.values())
+
+ def AddTestcase(self, testcase: "Testcase") -> None:
+ if testcase._parent is not None:
+ raise ValueError(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ if testcase._name in self._testcases:
+ raise DuplicateTestcaseException(f"Class already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent = self
+ self._testcases[testcase._name] = testcase
+
+ def AddTestcases(self, testcases: Iterable["Testcase"]) -> None:
+ for testcase in testcases:
+ self.AddTestcase(testcase)
+
+ def ToTestsuite(self) -> ut_Testsuite:
+ return ut_Testsuite(
+ self._name,
+ TestsuiteKind.Class,
+ # startTime=self._startTime,
+ # totalDuration=self._duration,
+ # status=self._status,
+ testcases=(tc.ToTestcase() for tc in self._testcases.values())
+ )
+
+ def ToTree(self) -> Node:
+ node = Node(
+ value=self._name,
+ children=(tc.ToTree() for tc in self._testcases.values())
+ )
+
+ return node
+
+ def __str__(self) -> str:
+ moduleName = self.__module__.split(".")[-1]
+ className = self.__class__.__name__
+ return (
+ f"<{moduleName}{className} {self._name}: {len(self._testcases)}>"
+ )
+
+
+@export
+class Testsuite(TestsuiteBase):
+ """
+ A testsuite is a mid-level element in the test entity hierarchy representing a logical group of tests.
+
+ Test suites contain test classes and are grouped by a test summary, which is the root of the hierarchy.
+ """
+
+ _hostname: str
+ _testclasses: Dict[str, "Testclass"]
+
+ def __init__(
+ self,
+ name: str,
+ hostname: Nullable[str] = None,
+ startTime: Nullable[datetime] = None,
+ duration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ testclasses: Nullable[Iterable["Testclass"]] = None,
+ parent: Nullable["TestsuiteSummary"] = None
+ ):
+ """
+ Initializes the fields of a test suite.
+
+ :param name: Name of the test suite.
+ :param startTime: Time when the test suite was started.
+ :param duration: duration of the entity's execution.
+ :param status: Overall status of the test suite.
+ :param parent: Reference to the parent test summary.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ :raises TypeError: If element in parameter 'testcases' is not a Testcase.
+ :raises AlreadyInHierarchyException: If a test case in parameter 'testcases' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If a test case in parameter 'testcases' is already listed (by name) in the list of test cases.
+ """
+ if parent is not None:
+ if not isinstance(parent, TestsuiteSummary):
+ ex = TypeError(f"Parameter 'parent' is not of type 'TestsuiteSummary'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testsuites[name] = self
+
+ super().__init__(name, startTime, duration, status, parent)
+
+ self._hostname = hostname
+
+ self._testclasses = {}
+ if testclasses is not None:
+ for testclass in testclasses:
+ if testclass._parent is not None:
+ raise ValueError(f"Class '{testclass._name}' is already part of a testsuite hierarchy.")
+
+ if testclass._name in self._testclasses:
+ raise DuplicateTestcaseException(f"Testsuite already contains a class with same name '{testclass._name}'.")
+
+ testclass._parent = self
+ self._testclasses[testclass._name] = testclass
+
+ @readonly
+ def Hostname(self) -> Nullable[str]:
+ return self._hostname
+
+ @readonly
+ def Testclasses(self) -> Dict[str, "Testclass"]:
+ return self._testclasses
+
+ @readonly
+ def TestclassCount(self) -> int:
+ return len(self._testclasses)
+
+ # @readonly
+ # def Testcases(self) -> Dict[str, "Testcase"]:
+ # return self._classes
+
+ @readonly
+ def TestcaseCount(self) -> int:
+ return sum(cls.TestcaseCount for cls in self._testclasses.values())
+
+ @readonly
+ def AssertionCount(self) -> int:
+ return sum(cls.AssertionCount for cls in self._testclasses.values())
+
+ def AddTestclass(self, testclass: "Testclass") -> None:
+ if testclass._parent is not None:
+ raise ValueError(f"Class '{testclass._name}' is already part of a testsuite hierarchy.")
+
+ if testclass._name in self._testclasses:
+ raise DuplicateTestcaseException(f"Testsuite already contains a class with same name '{testclass._name}'.")
+
+ testclass._parent = self
+ self._testclasses[testclass._name] = testclass
+
+ def AddTestclasses(self, testclasses: Iterable["Testclass"]) -> None:
+ for testcase in testclasses:
+ self.AddTestclass(testcase)
+
+ # def IterateTestsuites(self, scheme: IterationScheme = IterationScheme.TestsuiteDefault) -> Generator[TestsuiteType, None, None]:
+ # return self.Iterate(scheme)
+
+ def IterateTestcases(self, scheme: IterationScheme = IterationScheme.TestcaseDefault) -> Generator[Testcase, None, None]:
+ return self.Iterate(scheme)
+
+ def Copy(self) -> "Testsuite":
+ return self.__class__(
+ self._name,
+ self._hostname,
+ self._startTime,
+ self._duration,
+ self._status
+ )
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ for testclass in self._testclasses.values():
+ _ = testclass.Aggregate(strict)
+
+ tests += 1
+
+ status = testclass._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with state 'Unknown'.")
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testclass '{testclass._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testclass '{testclass._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[TestsuiteType, Testcase], None, None]:
+ """
+ Iterate the test suite and its child elements according to the iteration scheme.
+
+ If no scheme is given, use the default scheme.
+
+ :param scheme: Scheme how to iterate the test suite and its child elements.
+ :returns: A generator for iterating the results filtered and in the order defined by the iteration scheme.
+ """
+ assert IterationScheme.PreOrder | IterationScheme.PostOrder not in scheme
+
+ if IterationScheme.PreOrder in scheme:
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites in scheme:
+ yield self
+
+ if IterationScheme.IncludeTestcases in scheme:
+ for testcase in self._testclasses.values():
+ yield testcase
+
+ for testclass in self._testclasses.values():
+ yield from testclass.Iterate(scheme | IterationScheme.IncludeSelf)
+
+ if IterationScheme.PostOrder in scheme:
+ if IterationScheme.IncludeTestcases in scheme:
+ for testcase in self._testclasses.values():
+ yield testcase
+
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites in scheme:
+ yield self
+
+ @classmethod
+ def FromTestsuite(cls, testsuite: ut_Testsuite) -> "Testsuite":
+ """
+ Convert a test suite of the unified test entity data model to the JUnit specific data model's test suite object.
+
+ :param testsuite: Test suite from unified data model.
+ :return: Test suite from JUnit specific data model.
+ """
+ juTestsuite = cls(
+ testsuite._name,
+ startTime=testsuite._startTime,
+ duration=testsuite._totalDuration,
+ status= testsuite._status,
+ )
+
+ juTestsuite._tests = testsuite._tests
+ juTestsuite._skipped = testsuite._skipped
+ juTestsuite._errored = testsuite._errored
+ juTestsuite._failed = testsuite._failed
+ juTestsuite._passed = testsuite._passed
+
+ for tc in testsuite.IterateTestcases():
+ ts = tc._parent
+ if ts is None:
+ raise UnittestException(f"Testcase '{tc._name}' is not part of a hierarchy.")
+
+ classname = ts._name
+ ts = ts._parent
+ while ts is not None and ts._kind > TestsuiteKind.Logical:
+ classname = f"{ts._name}.{classname}"
+ ts = ts._parent
+
+ if classname in juTestsuite._testclasses:
+ juClass = juTestsuite._testclasses[classname]
+ else:
+ juClass = Testclass(classname, parent=juTestsuite)
+
+ juClass.AddTestcase(Testcase.FromTestcase(tc))
+
+ return juTestsuite
+
+ def ToTestsuite(self) -> ut_Testsuite:
+ testsuite = ut_Testsuite(
+ self._name,
+ TestsuiteKind.Logical,
+ startTime=self._startTime,
+ totalDuration=self._duration,
+ status=self._status,
+ )
+
+ for testclass in self._testclasses.values():
+ suite = testsuite
+ classpath = testclass._name.split(".")
+ for element in classpath:
+ if element in suite._testsuites:
+ suite = suite._testsuites[element]
+ else:
+ suite = ut_Testsuite(element, kind=TestsuiteKind.Package, parent=suite)
+
+ suite._kind = TestsuiteKind.Class
+ if suite._parent is not testsuite:
+ suite._parent._kind = TestsuiteKind.Module
+
+ suite.AddTestcases(tc.ToTestcase() for tc in testclass._testcases.values())
+
+ return testsuite
+
+ def ToTree(self) -> Node:
+ node = Node(
+ value=self._name,
+ children=(cls.ToTree() for cls in self._testclasses.values())
+ )
+ node["startTime"] = self._startTime
+ node["duration"] = self._duration
+
+ return node
+
+ def __str__(self) -> str:
+ moduleName = self.__module__.split(".")[-1]
+ className = self.__class__.__name__
+ return (
+ f"<{moduleName}{className} {self._name}: {self._status.name} - tests:{self._tests}>"
+ )
+
+
+@export
+class TestsuiteSummary(TestsuiteBase):
+ _testsuites: Dict[str, Testsuite]
+
+ def __init__(
+ self,
+ name: str,
+ startTime: Nullable[datetime] = None,
+ duration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ testsuites: Nullable[Iterable[Testsuite]] = None
+ ):
+ super().__init__(name, startTime, duration, status, None)
+
+ self._testsuites = {}
+ if testsuites is not None:
+ for testsuite in testsuites:
+ if testsuite._parent is not None:
+ raise ValueError(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ if testsuite._name in self._testsuites:
+ raise DuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent = self
+ self._testsuites[testsuite._name] = testsuite
+
+ @readonly
+ def Testsuites(self) -> Dict[str, Testsuite]:
+ return self._testsuites
+
+ @readonly
+ def TestcaseCount(self) -> int:
+ return sum(ts.TestcaseCount for ts in self._testsuites.values())
+
+ @readonly
+ def TestsuiteCount(self) -> int:
+ return len(self._testsuites)
+
+ @readonly
+ def AssertionCount(self) -> int:
+ return sum(ts.AssertionCount for ts in self._testsuites.values())
+
+ def AddTestsuite(self, testsuite: Testsuite) -> None:
+ if testsuite._parent is not None:
+ raise ValueError(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ if testsuite._name in self._testsuites:
+ raise DuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent = self
+ self._testsuites[testsuite._name] = testsuite
+
+ def AddTestsuites(self, testsuites: Iterable[Testsuite]) -> None:
+ for testsuite in testsuites:
+ self.AddTestsuite(testsuite)
+
+ def Aggregate(self) -> TestsuiteAggregateReturnType:
+ tests, skipped, errored, failed, passed = super().Aggregate()
+
+ self._tests = tests
+ self._skipped = skipped
+ self._errored = errored
+ self._failed = failed
+ self._passed = passed
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, skipped, errored, failed, passed
+
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[Testsuite, Testcase], None, None]:
+ """
+ Iterate the test suite summary and its child elements according to the iteration scheme.
+
+ If no scheme is given, use the default scheme.
+
+ :param scheme: Scheme how to iterate the test suite summary and its child elements.
+ :returns: A generator for iterating the results filtered and in the order defined by the iteration scheme.
+ """
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites | IterationScheme.PreOrder in scheme:
+ yield self
+
+ for testsuite in self._testsuites.values():
+ yield from testsuite.IterateTestsuites(scheme | IterationScheme.IncludeSelf)
+
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites | IterationScheme.PostOrder in scheme:
+ yield self
+
+ @classmethod
+ def FromTestsuiteSummary(cls, testsuiteSummary: ut_TestsuiteSummary) -> "TestsuiteSummary":
+ """
+ Convert a test suite summary of the unified test entity data model to the JUnit specific data model's test suite.
+
+ :param testsuiteSummary: Test suite summary from unified data model.
+ :return: Test suite summary from JUnit specific data model.
+ """
+ return cls(
+ testsuiteSummary._name,
+ startTime=testsuiteSummary._startTime,
+ duration=testsuiteSummary._totalDuration,
+ status=testsuiteSummary._status,
+ testsuites=(ut_Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+ )
+
+ def ToTestsuiteSummary(self) -> ut_TestsuiteSummary:
+ """
+ Convert this test suite summary a new test suite summary of the unified data model.
+
+ All fields are copied to the new instance. Child elements like test suites are copied recursively.
+
+ :return: A test suite summary of the unified test entity data model.
+ """
+ return ut_TestsuiteSummary(
+ self._name,
+ startTime=self._startTime,
+ totalDuration=self._duration,
+ status=self._status,
+ testsuites=(testsuite.ToTestsuite() for testsuite in self._testsuites.values())
+ )
+
+ def ToTree(self) -> Node:
+ node = Node(
+ value=self._name,
+ children=(ts.ToTree() for ts in self._testsuites.values())
+ )
+ node["startTime"] = self._startTime
+ node["duration"] = self._duration
+
+ return node
+
+ def __str__(self) -> str:
+ moduleName = self.__module__.split(".")[-1]
+ className = self.__class__.__name__
+ return (
+ f"<{moduleName}{className} {self._name}: {self._status.name} - tests:{self._tests}>"
+ )
+
+
+@export
+class Document(TestsuiteSummary, ut_Document):
+ _TESTCASE: ClassVar[Type[Testcase]] = Testcase
+ _TESTCLASS: ClassVar[Type[Testclass]] = Testclass
+ _TESTSUITE: ClassVar[Type[Testsuite]] = Testsuite
+
+ _readerMode: JUnitReaderMode
+ _xmlDocument: Nullable[_ElementTree]
+
+ def __init__(self, xmlReportFile: Path, analyzeAndConvert: bool = False, readerMode: JUnitReaderMode = JUnitReaderMode.Default):
+ super().__init__("Unprocessed JUnit XML file")
+
+ self._readerMode = readerMode
+ self._xmlDocument = None
+
+ ut_Document.__init__(self, xmlReportFile, analyzeAndConvert)
+
+ @classmethod
+ def FromTestsuiteSummary(cls, xmlReportFile: Path, testsuiteSummary: ut_TestsuiteSummary):
+ doc = cls(xmlReportFile)
+ doc._name = testsuiteSummary._name
+ doc._startTime = testsuiteSummary._startTime
+ doc._duration = testsuiteSummary._totalDuration
+ doc._status = testsuiteSummary._status
+ doc._tests = testsuiteSummary._tests
+ doc._skipped = testsuiteSummary._skipped
+ doc._errored = testsuiteSummary._errored
+ doc._failed = testsuiteSummary._failed
+ doc._passed = testsuiteSummary._passed
+
+ doc.AddTestsuites(Testsuite.FromTestsuite(testsuite) for testsuite in testsuiteSummary._testsuites.values())
+
+ return doc
+
+ def Analyze(self) -> None:
+ """
+ Analyze the XML file, parse the content into an XML data structure and validate the data structure using an XML
+ schema.
+
+ .. hint::
+
+ The time spend for analysis will be made available via property :data:`AnalysisDuration`.
+
+ The used XML schema definition is generic to support "any" dialect.
+ """
+ xmlSchemaFile = "Any-JUnit.xsd"
+ self._Analyze(xmlSchemaFile)
+
+ def _Analyze(self, xmlSchemaFile: str) -> None:
+ if not self._path.exists():
+ raise UnittestException(f"JUnit XML file '{self._path}' does not exist.") \
+ from FileNotFoundError(f"File '{self._path}' not found.")
+
+ startAnalysis = perf_counter_ns()
+ try:
+ xmlSchemaResourceFile = getResourceFile(Resources, xmlSchemaFile)
+ except ToolingException as ex:
+ raise UnittestException(f"Couldn't locate XML Schema '{xmlSchemaFile}' in package resources.") from ex
+
+ try:
+ schemaParser = XMLParser(ns_clean=True)
+ schemaRoot = parse(xmlSchemaResourceFile, schemaParser)
+ except XMLSyntaxError as ex:
+ raise UnittestException(f"XML Syntax Error while parsing XML Schema '{xmlSchemaFile}'.") from ex
+
+ try:
+ junitSchema = XMLSchema(schemaRoot)
+ except XMLSchemaParseError as ex:
+ raise UnittestException(f"Error while parsing XML Schema '{xmlSchemaFile}'.")
+
+ try:
+ junitParser = XMLParser(schema=junitSchema, ns_clean=True)
+ junitDocument = parse(self._path, parser=junitParser)
+
+ self._xmlDocument = junitDocument
+ except XMLSyntaxError as ex:
+ if version_info >= (3, 11): # pragma: no cover
+ for logEntry in junitParser.error_log:
+ ex.add_note(str(logEntry))
+ raise UnittestException(f"XML syntax or validation error for '{self._path}' using XSD schema '{xmlSchemaResourceFile}'.") from ex
+ except Exception as ex:
+ raise UnittestException(f"Couldn't open '{self._path}'.") from ex
+
+ endAnalysis = perf_counter_ns()
+ self._analysisDuration = (endAnalysis - startAnalysis) / 1e9
+
+ def Write(self, path: Nullable[Path] = None, overwrite: bool = False, regenerate: bool = False) -> None:
+ """
+ Write the data model as XML into a file adhering to the Any JUnit dialect.
+
+ :param path: Optional path to the XMl file, if internal path shouldn't be used.
+ :param overwrite: If true, overwrite an existing file.
+ :param regenerate: If true, regenerate the XML structure from data model.
+ :raises UnittestException: If the file cannot be overwritten.
+ :raises UnittestException: If the internal XML data structure wasn't generated.
+ :raises UnittestException: If the file cannot be opened or written.
+ """
+ if path is None:
+ path = self._path
+
+ if not overwrite and path.exists():
+ raise UnittestException(f"JUnit XML file '{path}' can not be overwritten.") \
+ from FileExistsError(f"File '{path}' already exists.")
+
+ if regenerate:
+ self.Generate(overwrite=True)
+
+ if self._xmlDocument is None:
+ ex = UnittestException(f"Internal XML document tree is empty and needs to be generated before write is possible.")
+ ex.add_note(f"Call 'JUnitDocument.Generate()' or 'JUnitDocument.Write(..., regenerate=True)'.")
+ raise ex
+
+ try:
+ with path.open("wb") as file:
+ file.write(tostring(self._xmlDocument, encoding="utf-8", xml_declaration=True, pretty_print=True))
+ except Exception as ex:
+ raise UnittestException(f"JUnit XML file '{path}' can not be written.") from ex
+
+ def Convert(self) -> None:
+ """
+ Convert the parsed and validated XML data structure into a JUnit test entity hierarchy.
+
+ This method converts the root element.
+
+ .. hint::
+
+ The time spend for model conversion will be made available via property :data:`ModelConversionDuration`.
+
+ :raises UnittestException: If XML was not read and parsed before.
+ """
+ if self._xmlDocument is None:
+ ex = UnittestException(f"JUnit XML file '{self._path}' needs to be read and analyzed by an XML parser.")
+ ex.add_note(f"Call 'JUnitDocument.Analyze()' or create the document using 'JUnitDocument(path, parse=True)'.")
+ raise ex
+
+ startConversion = perf_counter_ns()
+ rootElement: _Element = self._xmlDocument.getroot()
+
+ self._name = self._ConvertName(rootElement, optional=True)
+ self._startTime = self._ConvertTimestamp(rootElement, optional=True)
+ self._duration = self._ConvertTime(rootElement, optional=True)
+
+ if False: # self._readerMode is JUnitReaderMode.
+ self._tests = self._ConvertTests(testsuitesNode)
+ self._skipped = self._ConvertSkipped(testsuitesNode)
+ self._errored = self._ConvertErrors(testsuitesNode)
+ self._failed = self._ConvertFailures(testsuitesNode)
+ self._assertionCount = self._ConvertAssertions(testsuitesNode)
+
+ for rootNode in rootElement.iterchildren(tag="testsuite"): # type: _Element
+ self._ConvertTestsuite(self, rootNode)
+
+ if True: # self._readerMode is JUnitReaderMode.
+ self.Aggregate()
+
+ endConversation = perf_counter_ns()
+ self._modelConversion = (endConversation - startConversion) / 1e9
+
+ def _ConvertName(self, element: _Element, default: str = "root", optional: bool = True) -> str:
+ """
+ Convert the ``name`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``name`` attribute.
+ :param default: The default value, if no ``name`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``name`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``name`` attribute exists on the given element node.
+ """
+ if "name" in element.attrib:
+ return element.attrib["name"]
+ elif not optional:
+ raise UnittestException(f"Required parameter 'name' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertTimestamp(self, element: _Element, optional: bool = True) -> Nullable[datetime]:
+ """
+ Convert the ``timestamp`` attribute from an XML element node to a datetime.
+
+ :param element: The XML element node with a ``timestamp`` attribute.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``timestamp`` attribute's content if found, otherwise ``None``.
+ :raises UnittestException: If optional is false and no ``timestamp`` attribute exists on the given element node.
+ """
+ if "timestamp" in element.attrib:
+ timestamp = element.attrib["timestamp"]
+ return datetime.fromisoformat(timestamp)
+ elif not optional:
+ raise UnittestException(f"Required parameter 'timestamp' not found in tag '{element.tag}'.")
+ else:
+ return None
+
+ def _ConvertTime(self, element: _Element, optional: bool = True) -> Nullable[timedelta]:
+ """
+ Convert the ``time`` attribute from an XML element node to a timedelta.
+
+ :param element: The XML element node with a ``time`` attribute.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``time`` attribute's content if found, otherwise ``None``.
+ :raises UnittestException: If optional is false and no ``time`` attribute exists on the given element node.
+ """
+ if "time" in element.attrib:
+ time = element.attrib["time"]
+ return timedelta(seconds=float(time))
+ elif not optional:
+ raise UnittestException(f"Required parameter 'time' not found in tag '{element.tag}'.")
+ else:
+ return None
+
+ def _ConvertHostname(self, element: _Element, default: str = "localhost", optional: bool = True) -> str:
+ """
+ Convert the ``hostname`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``hostname`` attribute.
+ :param default: The default value, if no ``hostname`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``hostname`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``hostname`` attribute exists on the given element node.
+ """
+ if "hostname" in element.attrib:
+ return element.attrib["hostname"]
+ elif not optional:
+ raise UnittestException(f"Required parameter 'hostname' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertClassname(self, element: _Element) -> str:
+ """
+ Convert the ``classname`` attribute from an XML element node to a string.
+
+ :param element: The XML element node with a ``classname`` attribute.
+ :return: The ``classname`` attribute's content.
+ :raises UnittestException: If no ``classname`` attribute exists on the given element node.
+ """
+ if "classname" in element.attrib:
+ return element.attrib["classname"]
+ else:
+ raise UnittestException(f"Required parameter 'classname' not found in tag '{element.tag}'.")
+
+ def _ConvertTests(self, element: _Element, default: Nullable[int] = None, optional: bool = True) -> Nullable[int]:
+ """
+ Convert the ``tests`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``tests`` attribute.
+ :param default: The default value, if no ``tests`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``tests`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``tests`` attribute exists on the given element node.
+ """
+ if "tests" in element.attrib:
+ return int(element.attrib["tests"])
+ elif not optional:
+ raise UnittestException(f"Required parameter 'tests' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertSkipped(self, element: _Element, default: Nullable[int] = None, optional: bool = True) -> Nullable[int]:
+ """
+ Convert the ``skipped`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``skipped`` attribute.
+ :param default: The default value, if no ``skipped`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``skipped`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``skipped`` attribute exists on the given element node.
+ """
+ if "skipped" in element.attrib:
+ return int(element.attrib["skipped"])
+ elif not optional:
+ raise UnittestException(f"Required parameter 'skipped' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertErrors(self, element: _Element, default: Nullable[int] = None, optional: bool = True) -> Nullable[int]:
+ """
+ Convert the ``errors`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``errors`` attribute.
+ :param default: The default value, if no ``errors`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``errors`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``errors`` attribute exists on the given element node.
+ """
+ if "errors" in element.attrib:
+ return int(element.attrib["errors"])
+ elif not optional:
+ raise UnittestException(f"Required parameter 'errors' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertFailures(self, element: _Element, default: Nullable[int] = None, optional: bool = True) -> Nullable[int]:
+ """
+ Convert the ``failures`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``failures`` attribute.
+ :param default: The default value, if no ``failures`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``failures`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``failures`` attribute exists on the given element node.
+ """
+ if "failures" in element.attrib:
+ return int(element.attrib["failures"])
+ elif not optional:
+ raise UnittestException(f"Required parameter 'failures' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertAssertions(self, element: _Element, default: Nullable[int] = None, optional: bool = True) -> Nullable[int]:
+ """
+ Convert the ``assertions`` attribute from an XML element node to an integer.
+
+ :param element: The XML element node with a ``assertions`` attribute.
+ :param default: The default value, if no ``assertions`` attribute was found.
+ :param optional: If false, an exception is raised for the missing attribute.
+ :return: The ``assertions`` attribute's content if found, otherwise the given default value.
+ :raises UnittestException: If optional is false and no ``assertions`` attribute exists on the given element node.
+ """
+ if "assertions" in element.attrib:
+ return int(element.attrib["assertions"])
+ elif not optional:
+ raise UnittestException(f"Required parameter 'assertions' not found in tag '{element.tag}'.")
+ else:
+ return default
+
+ def _ConvertTestsuite(self, parent: TestsuiteSummary, testsuitesNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testsuite>`` to a test suite.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite summary as a parent element in the test entity hierarchy.
+ :param testsuitesNode: The current XML element node representing a test suite.
+ """
+ newTestsuite = self._TESTSUITE(
+ self._ConvertName(testsuitesNode, optional=False),
+ self._ConvertHostname(testsuitesNode, optional=True),
+ self._ConvertTimestamp(testsuitesNode, optional=True),
+ self._ConvertTime(testsuitesNode, optional=True),
+ parent=parent
+ )
+
+ if False: # self._readerMode is JUnitReaderMode.
+ self._tests = self._ConvertTests(testsuitesNode)
+ self._skipped = self._ConvertSkipped(testsuitesNode)
+ self._errored = self._ConvertErrors(testsuitesNode)
+ self._failed = self._ConvertFailures(testsuitesNode)
+ self._assertionCount = self._ConvertAssertions(testsuitesNode)
+
+ self._ConvertTestsuiteChildren(testsuitesNode, newTestsuite)
+
+ def _ConvertTestsuiteChildren(self, testsuitesNode: _Element, newTestsuite: Testsuite) -> None:
+ for node in testsuitesNode.iterchildren(): # type: _Element
+ # if node.tag == "testsuite":
+ # self._ConvertTestsuite(newTestsuite, node)
+ # el
+ if node.tag == "testcase":
+ self._ConvertTestcase(newTestsuite, node)
+
+ def _ConvertTestcase(self, parent: Testsuite, testcaseNode: _Element) -> None:
+ """
+ Convert the XML data structure of a ``<testcase>`` to a test case.
+
+ This method uses private helper methods provided by the base-class.
+
+ :param parent: The test suite as a parent element in the test entity hierarchy.
+ :param testcaseNode: The current XML element node representing a test case.
+ """
+ className = self._ConvertClassname(testcaseNode)
+ testclass = self._FindOrCreateTestclass(parent, className)
+
+ newTestcase = self._TESTCASE(
+ self._ConvertName(testcaseNode, optional=False),
+ self._ConvertTime(testcaseNode, optional=False),
+ assertionCount=self._ConvertAssertions(testcaseNode),
+ parent=testclass
+ )
+
+ self._ConvertTestcaseChildren(testcaseNode, newTestcase)
+
+ def _FindOrCreateTestclass(self, parent: Testsuite, className: str) -> Testclass:
+ if className in parent._testclasses:
+ return parent._testclasses[className]
+ else:
+ return self._TESTCLASS(className, parent=parent)
+
+ def _ConvertTestcaseChildren(self, testcaseNode: _Element, newTestcase: Testcase) -> None:
+ for node in testcaseNode.iterchildren(): # type: _Element
+ if isinstance(node, _Comment):
+ pass
+ elif isinstance(node, _Element):
+ if node.tag == "skipped":
+ newTestcase._status = TestcaseStatus.Skipped
+ elif node.tag == "failure":
+ newTestcase._status = TestcaseStatus.Failed
+ elif node.tag == "error":
+ newTestcase._status = TestcaseStatus.Errored
+ elif node.tag == "system-out":
+ pass
+ elif node.tag == "system-err":
+ pass
+ elif node.tag == "properties":
+ pass
+ else:
+ raise UnittestException(f"Unknown element '{node.tag}' in junit file.")
+ else:
+ pass
+
+ if newTestcase._status is TestcaseStatus.Unknown:
+ newTestcase._status = TestcaseStatus.Passed
+
+ def Generate(self, overwrite: bool = False) -> None:
+ """
+ Generate the internal XML data structure from test suites and test cases.
+
+ This method generates the XML root element (``<testsuites>``) and recursively calls other generated methods.
+
+ :param overwrite: Overwrite the internal XML data structure.
+ :raises UnittestException: If overwrite is false and the internal XML data structure is not empty.
+ """
+ if not overwrite and self._xmlDocument is not None:
+ raise UnittestException(f"Internal XML document is populated with data.")
+
+ rootElement = Element("testsuites")
+ rootElement.attrib["name"] = self._name
+ if self._startTime is not None:
+ rootElement.attrib["timestamp"] = f"{self._startTime.isoformat()}"
+ if self._duration is not None:
+ rootElement.attrib["time"] = f"{self._duration.total_seconds():.6f}"
+ rootElement.attrib["tests"] = str(self._tests)
+ rootElement.attrib["failures"] = str(self._failed)
+ rootElement.attrib["errors"] = str(self._errored)
+ rootElement.attrib["skipped"] = str(self._skipped)
+ # if self._assertionCount is not None:
+ # rootElement.attrib["assertions"] = f"{self._assertionCount}"
+
+ self._xmlDocument = ElementTree(rootElement)
+
+ for testsuite in self._testsuites.values():
+ self._GenerateTestsuite(testsuite, rootElement)
+
+ def _GenerateTestsuite(self, testsuite: Testsuite, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test suite.
+
+ This method generates the XML element (``<testsuite>``) and recursively calls other generated methods.
+
+ :param testsuite: The test suite to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testsuiteElement = SubElement(parentElement, "testsuite")
+ testsuiteElement.attrib["name"] = testsuite._name
+ if testsuite._startTime is not None:
+ testsuiteElement.attrib["timestamp"] = f"{testsuite._startTime.isoformat()}"
+ if testsuite._duration is not None:
+ testsuiteElement.attrib["time"] = f"{testsuite._duration.total_seconds():.6f}"
+ testsuiteElement.attrib["tests"] = str(testsuite._tests)
+ testsuiteElement.attrib["failures"] = str(testsuite._failed)
+ testsuiteElement.attrib["errors"] = str(testsuite._errored)
+ testsuiteElement.attrib["skipped"] = str(testsuite._skipped)
+ # if testsuite._assertionCount is not None:
+ # testsuiteElement.attrib["assertions"] = f"{testsuite._assertionCount}"
+ if testsuite._hostname is not None:
+ testsuiteElement.attrib["hostname"] = testsuite._hostname
+
+ for testclass in testsuite._testclasses.values():
+ for tc in testclass._testcases.values():
+ self._GenerateTestcase(tc, testsuiteElement)
+
+ def _GenerateTestcase(self, testcase: Testcase, parentElement: _Element) -> None:
+ """
+ Generate the internal XML data structure for a test case.
+
+ This method generates the XML element (``<testcase>``) and recursively calls other generated methods.
+
+ :param testcase: The test case to convert to an XML data structures.
+ :param parentElement: The parent XML data structure element, this data structure part will be added to.
+ :return:
+ """
+ testcaseElement = SubElement(parentElement, "testcase")
+ if testcase.Classname is not None:
+ testcaseElement.attrib["classname"] = testcase.Classname
+ testcaseElement.attrib["name"] = testcase._name
+ if testcase._duration is not None:
+ testcaseElement.attrib["time"] = f"{testcase._duration.total_seconds():.6f}"
+ if testcase._assertionCount is not None:
+ testcaseElement.attrib["assertions"] = f"{testcase._assertionCount}"
+
+ if testcase._status is TestcaseStatus.Passed:
+ pass
+ elif testcase._status is TestcaseStatus.Failed:
+ failureElement = SubElement(testcaseElement, "failure")
+ elif testcase._status is TestcaseStatus.Skipped:
+ skippedElement = SubElement(testcaseElement, "skipped")
+ else:
+ errorElement = SubElement(testcaseElement, "error")
+
+ def __str__(self) -> str:
+ moduleName = self.__module__.split(".")[-1]
+ className = self.__class__.__name__
+ return (
+ f"<{moduleName}{className} {self._name} ({self._path}): {self._status.name} - suites/tests:{self.TestsuiteCount}/{self.TestcaseCount}>"
+ )
+
# ==================================================================================================================== #
+# _____ ____ _ _ ____ _ #
+# _ __ _ _| ____| _ \ / \ / \ | _ \ ___ _ __ ___ _ __| |_ ___ #
+# | '_ \| | | | _| | | | |/ _ \ / _ \ | |_) / _ \ '_ \ / _ \| '__| __/ __| #
+# | |_) | |_| | |___| |_| / ___ \ / ___ \ _| _ < __/ |_) | (_) | | | |_\__ \ #
+# | .__/ \__, |_____|____/_/ \_\/_/ \_(_)_| \_\___| .__/ \___/|_| \__|___/ #
+# |_| |___/ |_| #
+# ==================================================================================================================== #
+# Authors: #
+# Patrick Lehmann #
+# #
+# License: #
+# ==================================================================================================================== #
+# Copyright 2024-2024 Electronic Design Automation Abstraction (EDA²) #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you may not use this file except in compliance with the License. #
+# You may obtain a copy of the License at #
+# #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, #
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+# #
+# SPDX-License-Identifier: Apache-2.0 #
+# ==================================================================================================================== #
+#
+"""
+The pyEDAA.Reports.Unittesting package implements a hierarchy of test entities. These are test cases, test suites and a
+test summary provided as a class hierarchy. Test cases are the leaf elements in the hierarchy and abstract an
+individual test run. Test suites are used to group multiple test cases or other test suites. The root element is a test
+summary. When such a summary is stored in a file format like Ant + JUnit4 XML, a file format specific document is
+derived from a summary class.
+
+**Data Model**
+
+.. mermaid::
+
+ graph TD;
+ doc[Document]
+ sum[Summary]
+ ts1[Testsuite]
+ ts2[Testsuite]
+ ts21[Testsuite]
+ tc11[Testcase]
+ tc12[Testcase]
+ tc13[Testcase]
+ tc21[Testcase]
+ tc22[Testcase]
+ tc211[Testcase]
+ tc212[Testcase]
+ tc213[Testcase]
+
+ doc:::root -.-> sum:::summary
+ sum --> ts1:::suite
+ sum --> ts2:::suite
+ ts2 --> ts21:::suite
+ ts1 --> tc11:::case
+ ts1 --> tc12:::case
+ ts1 --> tc13:::case
+ ts2 --> tc21:::case
+ ts2 --> tc22:::case
+ ts21 --> tc211:::case
+ ts21 --> tc212:::case
+ ts21 --> tc213:::case
+
+ classDef root fill:#4dc3ff
+ classDef summary fill:#80d4ff
+ classDef suite fill:#b3e6ff
+ classDef case fill:#eeccff
+"""
+from datetime import timedelta, datetime
+from enum import Flag, IntEnum
+from pathlib import Path
+from sys import version_info
+from typing import Optional as Nullable, Dict, Iterable, Any, Tuple, Generator, Union, List, Generic, TypeVar, Mapping
+
+from pyTooling.Common import getFullyQualifiedName
+from pyTooling.Decorators import export, readonly
+from pyTooling.MetaClasses import ExtendedType, abstractmethod
+from pyTooling.Tree import Node
+
+from pyEDAA.Reports import ReportException
+
+
+@export
+class UnittestException(ReportException):
+ """Base-exception for all unit test related exceptions."""
+
+
+@export
+class AlreadyInHierarchyException(UnittestException):
+ """
+ A unit test exception raised if the element is already part of a hierarchy.
+
+ This exception is caused by an inconsistent data model. Elements added to the hierarchy should be part of the same
+ hierarchy should occur only once in the hierarchy.
+
+ .. hint::
+
+ This is usually caused by a non-None parent reference.
+ """
+
+
+@export
+class DuplicateTestsuiteException(UnittestException):
+ """
+ A unit test exception raised on duplicate test suites (by name).
+
+ This exception is raised, if a child test suite with same name already exist in the test suite.
+
+ .. hint::
+
+ Test suite names need to be unique per parent element (test suite or test summary).
+ """
+
+
+@export
+class DuplicateTestcaseException(UnittestException):
+ """
+ A unit test exception raised on duplicate test cases (by name).
+
+ This exception is raised, if a child test case with same name already exist in the test suite.
+
+ .. hint::
+
+ Test case names need to be unique per parent element (test suite).
+ """
+
+
+@export
+class TestcaseStatus(Flag):
+ """A flag enumeration describing the status of a test case."""
+ Unknown = 0 #: Testcase status is uninitialized and therefore unknown.
+ Excluded = 1 #: Testcase was permanently excluded / disabled
+ Skipped = 2 #: Testcase was temporarily skipped (e.g. based on a condition)
+ Weak = 4 #: No assertions were recorded.
+ Passed = 8 #: A passed testcase, because all assertions were successful.
+ Failed = 16 #: A failed testcase due to at least one failed assertion.
+
+ Mask = Excluded | Skipped | Weak | Passed | Failed
+
+ Inverted = 128 #: To mark inverted results
+ UnexpectedPassed = Failed | Inverted
+ ExpectedFailed = Passed | Inverted
+
+ Warned = 1024 #: Runtime warning
+ Errored = 2048 #: Runtime error (mostly caught exceptions)
+ Aborted = 4096 #: Uncaught runtime exception
+
+ SetupError = 8192 #: Preparation / compilation error
+ TearDownError = 16384 #: Cleanup error / resource release error
+ Inconsistent = 32768 #: Dataset is inconsistent
+
+ Flags = Warned | Errored | Aborted | SetupError | TearDownError | Inconsistent
+
+ # TODO: timed out ?
+
+ def __matmul__(self, other: "TestcaseStatus") -> "TestcaseStatus":
+ s = self & self.Mask
+ o = other & self.Mask
+ if s is self.Excluded:
+ resolved = self.Excluded if o is self.Excluded else self.Unknown
+ elif s is self.Skipped:
+ resolved = self.Unknown if (o is self.Unknown) or (o is self.Excluded) else o
+ elif s is self.Weak:
+ resolved = self.Weak if o is self.Weak else self.Unknown
+ elif s is self.Passed:
+ resolved = self.Passed if (o is self.Skipped) or (o is self.Passed) else self.Unknown
+ elif s is self.Failed:
+ resolved = self.Failed if (o is self.Skipped) or (o is self.Failed) else self.Unknown
+ else:
+ resolved = self.Unknown
+
+ resolved |= (self & self.Flags) | (other & self.Flags)
+ return resolved
+
+
+@export
+class TestsuiteStatus(Flag):
+ """A flag enumeration describing the status of a test suite."""
+ Unknown = 0
+ Excluded = 1 #: Testcase was permanently excluded / disabled
+ Skipped = 2 #: Testcase was temporarily skipped (e.g. based on a condition)
+ Empty = 4 #: No tests in suite
+ Passed = 8 #: Passed testcase, because all assertions succeeded
+ Failed = 16 #: Failed testcase due to failing assertions
+
+ Mask = Excluded | Skipped | Empty | Passed | Failed
+
+ Inverted = 128 #: To mark inverted results
+ UnexpectedPassed = Failed | Inverted
+ ExpectedFailed = Passed | Inverted
+
+ Warned = 1024 #: Runtime warning
+ Errored = 2048 #: Runtime error (mostly caught exceptions)
+ Aborted = 4096 #: Uncaught runtime exception
+
+ SetupError = 8192 #: Preparation / compilation error
+ TearDownError = 16384 #: Cleanup error / resource release error
+
+ Flags = Warned | Errored | Aborted | SetupError | TearDownError
+
+
+@export
+class TestsuiteKind(IntEnum):
+ """Enumeration describing the kind of test suite."""
+ Root = 0 #: Root element of the hierarchy.
+ Logical = 1 #: Represents a logical unit.
+ Namespace = 2 #: Represents a namespace.
+ Package = 3 #: Represents a package.
+ Module = 4 #: Represents a module.
+ Class = 5 #: Represents a class.
+
+
+@export
+class IterationScheme(Flag):
+ """
+ A flag enumeration for selecting the test suite iteration scheme.
+
+ When a test entity hierarchy is (recursively) iterated, this iteration scheme describes how to iterate the hierarchy
+ and what elements to return as a result.
+ """
+ Unknown = 0 #: Neutral element.
+ IncludeSelf = 1 #: Also include the element itself.
+ IncludeTestsuites = 2 #: Include test suites into the result.
+ IncludeTestcases = 4 #: Include test cases into the result.
+
+ Recursive = 8 #: Iterate recursively.
+
+ PreOrder = 16 #: Iterate in pre-order (top-down: current node, then child element left-to-right).
+ PostOrder = 32 #: Iterate in pre-order (bottom-up: child element left-to-right, then current node).
+
+ Default = IncludeTestsuites | Recursive | IncludeTestcases | PreOrder #: Recursively iterate all test entities in pre-order.
+ TestsuiteDefault = IncludeTestsuites | Recursive | PreOrder #: Recursively iterate only test suites in pre-order.
+ TestcaseDefault = IncludeTestcases | Recursive | PreOrder #: Recursively iterate only test cases in pre-order.
+
+
+TestsuiteType = TypeVar("TestsuiteType", bound="Testsuite")
+TestcaseAggregateReturnType = Tuple[int, int, int, timedelta]
+TestsuiteAggregateReturnType = Tuple[int, int, int, int, int, int, int, int, int, int, int, timedelta]
+
+
+@export
+class Base(metaclass=ExtendedType, slots=True):
+ """
+ Base-class for all test entities (test cases, test suites, ...).
+
+ It provides a reference to the parent test entity, so bidirectional referencing can be used in the test entity
+ hierarchy.
+
+ Every test entity has a name to identity it. It's also used in the parent's child element dictionaries to identify the
+ child. |br|
+ E.g. it's used as a test case name in the dictionary of test cases in a test suite.
+
+ Every test entity has fields for time tracking. If known, a start time and a test duration can be set. For more
+ details, a setup duration and teardown duration can be added. All durations are summed up in a total duration field.
+
+ As tests can have warnings and errors or even fail, these messages are counted and aggregated in the test entity
+ hierarchy.
+
+ Every test entity offers an internal dictionary for annotations. |br|
+ This feature is for example used by Ant + JUnit4's XML property fields.
+ """
+
+ _parent: Nullable["TestsuiteBase"]
+ _name: str
+
+ _startTime: Nullable[datetime]
+ _setupDuration: Nullable[timedelta]
+ _testDuration: Nullable[timedelta]
+ _teardownDuration: Nullable[timedelta]
+ _totalDuration: Nullable[timedelta]
+
+ _warningCount: int
+ _errorCount: int
+ _fatalCount: int
+
+ _dict: Dict[str, Any]
+
+ def __init__(
+ self,
+ name: str,
+ startTime: Nullable[datetime] = None,
+ setupDuration: Nullable[timedelta] = None,
+ testDuration: Nullable[timedelta] = None,
+ teardownDuration: Nullable[timedelta] = None,
+ totalDuration: Nullable[timedelta] = None,
+ warningCount: int = 0,
+ errorCount: int = 0,
+ fatalCount: int = 0,
+ keyValuePairs: Nullable[Mapping[str, Any]] = None,
+ parent: Nullable["TestsuiteBase"] = None
+ ):
+ """
+ Initializes the fields of the base-class.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of the entity's test run.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown).
+ :param warningCount: Count of encountered warnings.
+ :param errorCount: Count of encountered errors.
+ :param fatalCount: Count of encountered fatal errors.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test entity with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ :raises ValueError: If parameter 'name' is None.
+ :raises TypeError: If parameter 'name' is not a string.
+ :raises ValueError: If parameter 'name' is empty.
+ :raises TypeError: If parameter 'testDuration' is not a timedelta.
+ :raises TypeError: If parameter 'setupDuration' is not a timedelta.
+ :raises TypeError: If parameter 'teardownDuration' is not a timedelta.
+ :raises TypeError: If parameter 'totalDuration' is not a timedelta.
+ :raises TypeError: If parameter 'warningCount' is not an integer.
+ :raises TypeError: If parameter 'errorCount' is not an integer.
+ :raises TypeError: If parameter 'fatalCount' is not an integer.
+ :raises TypeError: If parameter 'keyValuePairs' is not a Mapping.
+ :raises ValueError: If parameter 'totalDuration' is not consistent.
+ """
+
+ if parent is not None and not isinstance(parent, TestsuiteBase):
+ ex = TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ if name is None:
+ raise ValueError(f"Parameter 'name' is None.")
+ elif not isinstance(name, str):
+ ex = TypeError(f"Parameter 'name' is not of type 'str'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(name)}'.")
+ raise ex
+ elif name.strip() == "":
+ raise ValueError(f"Parameter 'name' is empty.")
+
+ self._parent = parent
+ self._name = name
+
+ if testDuration is not None and not isinstance(testDuration, timedelta):
+ ex = TypeError(f"Parameter 'testDuration' is not of type 'timedelta'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testDuration)}'.")
+ raise ex
+
+ if setupDuration is not None and not isinstance(setupDuration, timedelta):
+ ex = TypeError(f"Parameter 'setupDuration' is not of type 'timedelta'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(setupDuration)}'.")
+ raise ex
+
+ if teardownDuration is not None and not isinstance(teardownDuration, timedelta):
+ ex = TypeError(f"Parameter 'teardownDuration' is not of type 'timedelta'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(teardownDuration)}'.")
+ raise ex
+
+ if totalDuration is not None and not isinstance(totalDuration, timedelta):
+ ex = TypeError(f"Parameter 'totalDuration' is not of type 'timedelta'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(totalDuration)}'.")
+ raise ex
+
+ if testDuration is not None:
+ if setupDuration is not None:
+ if teardownDuration is not None:
+ if totalDuration is not None:
+ if totalDuration < (setupDuration + testDuration + teardownDuration):
+ raise ValueError(f"Parameter 'totalDuration' can not be less than the sum of setup, test and teardown durations.")
+ else: # no total
+ totalDuration = setupDuration + testDuration + teardownDuration
+ # no teardown
+ elif totalDuration is not None:
+ if totalDuration < (setupDuration + testDuration):
+ raise ValueError(f"Parameter 'totalDuration' can not be less than the sum of setup and test durations.")
+ # no teardown, no total
+ else:
+ totalDuration = setupDuration + testDuration
+ # no setup
+ elif teardownDuration is not None:
+ if totalDuration is not None:
+ if totalDuration < (testDuration + teardownDuration):
+ raise ValueError(f"Parameter 'totalDuration' can not be less than the sum of test and teardown durations.")
+ else: # no setup, no total
+ totalDuration = testDuration + teardownDuration
+ # no setup, no teardown
+ elif totalDuration is not None:
+ if totalDuration < testDuration:
+ raise ValueError(f"Parameter 'totalDuration' can not be less than test durations.")
+ else: # no setup, no teardown, no total
+ totalDuration = testDuration
+ # no test
+ elif totalDuration is not None:
+ testDuration = totalDuration
+ if setupDuration is not None:
+ testDuration -= setupDuration
+ if teardownDuration is not None:
+ testDuration -= teardownDuration
+
+ self._startTime = startTime
+ self._setupDuration = setupDuration
+ self._testDuration = testDuration
+ self._teardownDuration = teardownDuration
+ self._totalDuration = totalDuration
+
+ if not isinstance(warningCount, int):
+ ex = TypeError(f"Parameter 'warningCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(warningCount)}'.")
+ raise ex
+
+ if not isinstance(errorCount, int):
+ ex = TypeError(f"Parameter 'errorCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(errorCount)}'.")
+ raise ex
+
+ if not isinstance(fatalCount, int):
+ ex = TypeError(f"Parameter 'fatalCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(fatalCount)}'.")
+ raise ex
+
+ self._warningCount = warningCount
+ self._errorCount = errorCount
+ self._fatalCount = fatalCount
+
+ if keyValuePairs is not None and not isinstance(keyValuePairs, Mapping):
+ ex = TypeError(f"Parameter 'keyValuePairs' is not a mapping.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(keyValuePairs)}'.")
+ raise ex
+
+ self._dict = {} if keyValuePairs is None else {k: v for k, v in keyValuePairs}
+
+ # QUESTION: allow Parent as setter?
+ @readonly
+ def Parent(self) -> Nullable["TestsuiteBase"]:
+ """
+ Read-only property returning the reference to the parent test entity.
+
+ :return: Reference to the parent entity.
+ """
+ return self._parent
+
+ @readonly
+ def Name(self) -> str:
+ """
+ Read-only property returning the test entity's name.
+
+ :return:
+ """
+ return self._name
+
+ @readonly
+ def StartTime(self) -> Nullable[datetime]:
+ """
+ Read-only property returning the time when the test entity was started.
+
+ :return: Time when the test entity was started.
+ """
+ return self._startTime
+
+ @readonly
+ def SetupDuration(self) -> Nullable[timedelta]:
+ """
+ Read-only property returning the duration of the test entity's setup.
+
+ :return: Duration it took to set up the entity.
+ """
+ return self._setupDuration
+
+ @readonly
+ def TestDuration(self) -> Nullable[timedelta]:
+ """
+ Read-only property returning the duration of a test entities run.
+
+ This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not
+ distinguishable, assign setup and teardown durations with zero.
+
+ :return: Duration of the entity's test run.
+ """
+ return self._testDuration
+
+ @readonly
+ def TeardownDuration(self) -> Nullable[timedelta]:
+ """
+ Read-only property returning the duration of the test entity's teardown.
+
+ :return: Duration it took to tear down the entity.
+ """
+ return self._teardownDuration
+
+ @readonly
+ def TotalDuration(self) -> Nullable[timedelta]:
+ """
+ Read-only property returning the total duration of a test entity run.
+
+ this duration includes setup and teardown durations.
+
+ :return: Total duration of the entity's execution (setup + test + teardown)
+ """
+ return self._totalDuration
+
+ @readonly
+ def WarningCount(self) -> int:
+ """
+ Read-only property returning the number of encountered warnings.
+
+ :return: Count of encountered warnings.
+ """
+ return self._warningCount
+
+ @readonly
+ def ErrorCount(self) -> int:
+ """
+ Read-only property returning the number of encountered errors.
+
+ :return: Count of encountered errors.
+ """
+ return self._errorCount
+
+ @readonly
+ def FatalCount(self) -> int:
+ """
+ Read-only property returning the number of encountered fatal errors.
+
+ :return: Count of encountered fatal errors.
+ """
+ return self._fatalCount
+
+ def __len__(self) -> int:
+ """
+ Returns the number of annotated key-value pairs.
+
+ :return: Number of annotated key-value pairs.
+ """
+ return len(self._dict)
+
+ def __getitem__(self, key: str) -> Any:
+ """
+ Access a key-value pair by key.
+
+ :param key: Name if the key-value pair.
+ :return: Value of the accessed key.
+ """
+ return self._dict[key]
+
+ def __setitem__(self, key: str, value: Any) -> None:
+ """
+ Set the value of a key-value pair by key.
+
+ If the pair doesn't exist yet, it's created.
+
+ :param key: Key of the key-value pair.
+ :param value: Value of the key-value pair.
+ """
+ self._dict[key] = value
+
+ def __delitem__(self, key: str) -> None:
+ """
+ Delete a key-value pair by key.
+
+ :param key: Name if the key-value pair.
+ """
+ del self._dict[key]
+
+ def __contains__(self, key: str) -> bool:
+ """
+ Returns True, if a key-value pairs was annotated by this key.
+
+ :param key: Name of the key-value pair.
+ :return: True, if the pair was annotated.
+ """
+ return key in self._dict
+
+ def __iter__(self) -> Generator[Tuple[str, Any], None, None]:
+ """
+ Iterate all annotated key-value pairs.
+
+ :return: A generator of key-value pair tuples (key, value).
+ """
+ yield from self._dict.items()
+
+ @abstractmethod
+ def Aggregate(self, strict: bool = True):
+ """
+ Aggregate all test entities in the hierarchy.
+
+ :return:
+ """
+
+ @abstractmethod
+ def __str__(self) -> str:
+ """
+ Formats the test entity as human-readable incl. some statistics.
+ """
+
+
+@export
+class Testcase(Base):
+ """
+ A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.
+
+ Test cases are grouped by test suites in the test entity hierarchy. The root of the hierarchy is a test summary.
+
+ Every test case has an overall status like unknown, skipped, failed or passed.
+
+ In addition to all features from its base-class, test cases provide additional statistics for passed and failed
+ assertions (checks) as well as a sum thereof.
+ """
+
+ _status: TestcaseStatus
+ _assertionCount: Nullable[int]
+ _failedAssertionCount: Nullable[int]
+ _passedAssertionCount: Nullable[int]
+
+ def __init__(
+ self,
+ name: str,
+ startTime: Nullable[datetime] = None,
+ setupDuration: Nullable[timedelta] = None,
+ testDuration: Nullable[timedelta] = None,
+ teardownDuration: Nullable[timedelta] = None,
+ totalDuration: Nullable[timedelta] = None,
+ status: TestcaseStatus = TestcaseStatus.Unknown,
+ assertionCount: Nullable[int] = None,
+ failedAssertionCount: Nullable[int] = None,
+ passedAssertionCount: Nullable[int] = None,
+ warningCount: int = 0,
+ errorCount: int = 0,
+ fatalCount: int = 0,
+ keyValuePairs: Nullable[Mapping[str, Any]] = None,
+ parent: Nullable["Testsuite"] = None
+ ):
+ """
+ Initializes the fields of a test case.
+
+ :param name: Name of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of the entity's test run.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Status of the test case.
+ :param assertionCount: Number of assertions within the test.
+ :param failedAssertionCount: Number of failed assertions within the test.
+ :param passedAssertionCount: Number of passed assertions within the test.
+ :param warningCount: Count of encountered warnings.
+ :param errorCount: Count of encountered errors.
+ :param fatalCount: Count of encountered fatal errors.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test case.
+ :param parent: Reference to the parent test suite.
+ :raises TypeError: If parameter 'parent' is not a Testsuite.
+ :raises ValueError: If parameter 'assertionCount' is not consistent.
+ """
+
+ if parent is not None:
+ if not isinstance(parent, Testsuite):
+ ex = TypeError(f"Parameter 'parent' is not of type 'Testsuite'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testcases[name] = self
+
+ super().__init__(
+ name,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ warningCount,
+ errorCount,
+ fatalCount,
+ keyValuePairs,
+ parent
+ )
+
+ if not isinstance(status, TestcaseStatus):
+ ex = TypeError(f"Parameter 'status' is not of type 'TestcaseStatus'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(status)}'.")
+ raise ex
+
+ self._status = status
+
+ if assertionCount is not None and not isinstance(assertionCount, int):
+ ex = TypeError(f"Parameter 'assertionCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(assertionCount)}'.")
+ raise ex
+
+ if failedAssertionCount is not None and not isinstance(failedAssertionCount, int):
+ ex = TypeError(f"Parameter 'failedAssertionCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(failedAssertionCount)}'.")
+ raise ex
+
+ if passedAssertionCount is not None and not isinstance(passedAssertionCount, int):
+ ex = TypeError(f"Parameter 'passedAssertionCount' is not of type 'int'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(passedAssertionCount)}'.")
+ raise ex
+
+ self._assertionCount = assertionCount
+ if assertionCount is not None:
+ if failedAssertionCount is not None:
+ self._failedAssertionCount = failedAssertionCount
+
+ if passedAssertionCount is not None:
+ if passedAssertionCount + failedAssertionCount != assertionCount:
+ raise ValueError(f"passed assertion count ({passedAssertionCount}) + failed assertion count ({failedAssertionCount} != assertion count ({assertionCount}")
+
+ self._passedAssertionCount = passedAssertionCount
+ else:
+ self._passedAssertionCount = assertionCount - failedAssertionCount
+ elif passedAssertionCount is not None:
+ self._passedAssertionCount = passedAssertionCount
+ self._failedAssertionCount = assertionCount - passedAssertionCount
+ else:
+ raise ValueError(f"Neither passed assertion count nor failed assertion count are provided.")
+ elif failedAssertionCount is not None:
+ self._failedAssertionCount = failedAssertionCount
+
+ if passedAssertionCount is not None:
+ self._passedAssertionCount = passedAssertionCount
+ self._assertionCount = passedAssertionCount + failedAssertionCount
+ else:
+ raise ValueError(f"Passed assertion count is mandatory, if failed assertion count is provided instead of assertion count.")
+ elif passedAssertionCount is not None:
+ raise ValueError(f"Assertion count or failed assertion count is mandatory, if passed assertion count is provided.")
+ else:
+ self._passedAssertionCount = None
+ self._failedAssertionCount = None
+
+ @readonly
+ def Status(self) -> TestcaseStatus:
+ """
+ Read-only property returning the status of the test case.
+
+ :return: The test case's status.
+ """
+ return self._status
+
+ @readonly
+ def AssertionCount(self) -> int:
+ """
+ Read-only property returning the number of assertions (checks) in a test case.
+
+ :return: Number of assertions.
+ """
+ if self._assertionCount is None:
+ return 0
+ return self._assertionCount
+
+ @readonly
+ def FailedAssertionCount(self) -> int:
+ """
+ Read-only property returning the number of failed assertions (failed checks) in a test case.
+
+ :return: Number of assertions.
+ """
+ return self._failedAssertionCount
+
+ @readonly
+ def PassedAssertionCount(self) -> int:
+ """
+ Read-only property returning the number of passed assertions (successful checks) in a test case.
+
+ :return: Number of passed assertions.
+ """
+ return self._passedAssertionCount
+
+ def Copy(self) -> "Testcase":
+ return self.__class__(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._testDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount,
+ )
+
+ def Aggregate(self, strict: bool = True) -> TestcaseAggregateReturnType:
+ if self._status is TestcaseStatus.Unknown:
+ if self._assertionCount is None:
+ self._status = TestcaseStatus.Passed
+ elif self._assertionCount == 0:
+ self._status = TestcaseStatus.Weak
+ elif self._failedAssertionCount == 0:
+ self._status = TestcaseStatus.Passed
+ else:
+ self._status = TestcaseStatus.Failed
+
+ if self._warningCount > 0:
+ self._status |= TestcaseStatus.Warned
+
+ if self._errorCount > 0:
+ self._status |= TestcaseStatus.Errored
+
+ if self._fatalCount > 0:
+ self._status |= TestcaseStatus.Aborted
+
+ if strict:
+ self._status = self._status & ~TestcaseStatus.Passed | TestcaseStatus.Failed
+
+ # TODO: check for setup errors
+ # TODO: check for teardown errors
+
+ totalDuration = timedelta() if self._totalDuration is None else self._totalDuration
+
+ return self._warningCount, self._errorCount, self._fatalCount, totalDuration
+
+ def __str__(self) -> str:
+ """
+ Formats the test case as human-readable incl. statistics.
+
+ :pycode:`f"<Testcase {}: {} - assert/pass/fail:{}/{}/{} - warn/error/fatal:{}/{}/{} - setup/test/teardown:{}/{}/{}>"`
+
+ :return: Human-readable summary of a test case object.
+ """
+ return (
+ f"<Testcase {self._name}: {self._status.name} -"
+ f" assert/pass/fail:{self._assertionCount}/{self._passedAssertionCount}/{self._failedAssertionCount} -"
+ f" warn/error/fatal:{self._warningCount}/{self._errorCount}/{self._fatalCount} -"
+ f" setup/test/teardown:{self._setupDuration:.3f}/{self._testDuration:.3f}/{self._teardownDuration:.3f}>"
+ )
+
+
+@export
+class TestsuiteBase(Base, Generic[TestsuiteType]):
+ """
+ Base-class for all test suites and for test summaries.
+
+ A test suite is a mid-level grouping element in the test entity hierarchy, whereas the test summary is the root
+ element in that hierarchy. While a test suite groups other test suites and test cases, a test summary can only group
+ test suites. Thus, a test summary contains no test cases.
+ """
+
+ _kind: TestsuiteKind
+ _status: TestsuiteStatus
+ _testsuites: Dict[str, TestsuiteType]
+
+ _tests: int
+ _inconsistent: int
+ _excluded: int
+ _skipped: int
+ _errored: int
+ _weak: int
+ _failed: int
+ _passed: int
+
+ def __init__(
+ self,
+ name: str,
+ kind: TestsuiteKind = TestsuiteKind.Logical,
+ startTime: Nullable[datetime] = None,
+ setupDuration: Nullable[timedelta] = None,
+ testDuration: Nullable[timedelta] = None,
+ teardownDuration: Nullable[timedelta] = None,
+ totalDuration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ warningCount: int = 0,
+ errorCount: int = 0,
+ fatalCount: int = 0,
+ testsuites: Nullable[Iterable[TestsuiteType]] = None,
+ keyValuePairs: Nullable[Mapping[str, Any]] = None,
+ parent: Nullable["Testsuite"] = None
+ ):
+ """
+ Initializes the based-class fields of a test suite or test summary.
+
+ :param name: Name of the test entity.
+ :param kind: Kind of the test entity.
+ :param startTime: Time when the test entity was started.
+ :param setupDuration: Duration it took to set up the entity.
+ :param testDuration: Duration of all tests listed in the test entity.
+ :param teardownDuration: Duration it took to tear down the entity.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test entity.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test entity with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test entity with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'parent' is not a TestsuiteBase.
+ :raises TypeError: If parameter 'testsuites' is not iterable.
+ :raises TypeError: If element in parameter 'testsuites' is not a Testsuite.
+ :raises AlreadyInHierarchyException: If a test suite in parameter 'testsuites' is already part of a test entity hierarchy.
+ :raises DuplicateTestsuiteException: If a test suite in parameter 'testsuites' is already listed (by name) in the list of test suites.
+ """
+ if parent is not None:
+ if not isinstance(parent, TestsuiteBase):
+ ex = TypeError(f"Parameter 'parent' is not of type 'TestsuiteBase'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(parent)}'.")
+ raise ex
+
+ parent._testsuites[name] = self
+
+ super().__init__(
+ name,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ warningCount,
+ errorCount,
+ fatalCount,
+ keyValuePairs,
+ parent
+ )
+
+ self._kind = kind
+ self._status = status
+
+ self._testsuites = {}
+ if testsuites is not None:
+ if not isinstance(testsuites, Iterable):
+ ex = TypeError(f"Parameter 'testsuites' is not iterable.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuites)}'.")
+ raise ex
+
+ for testsuite in testsuites:
+ if not isinstance(testsuite, Testsuite):
+ ex = TypeError(f"Element of parameter 'testsuites' is not of type 'Testsuite'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuite)}'.")
+ raise ex
+
+ if testsuite._parent is not None:
+ raise AlreadyInHierarchyException(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ if testsuite._name in self._testsuites:
+ raise DuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent = self
+ self._testsuites[testsuite._name] = testsuite
+
+ self._status = TestsuiteStatus.Unknown
+ self._tests = 0
+ self._inconsistent = 0
+ self._excluded = 0
+ self._skipped = 0
+ self._errored = 0
+ self._weak = 0
+ self._failed = 0
+ self._passed = 0
+
+ @readonly
+ def Kind(self) -> TestsuiteKind:
+ """
+ Read-only property returning the kind of the test suite.
+
+ Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests
+ grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming
+ language construct.
+
+ Test summaries always return kind ``Root``.
+
+ :return: Kind of the test suite.
+ """
+ return self._kind
+
+ @readonly
+ def Status(self) -> TestsuiteStatus:
+ """
+ Read-only property returning the aggregated overall status of the test suite.
+
+ :return: Overall status of the test suite.
+ """
+ return self._status
+
+ @readonly
+ def Testsuites(self) -> Dict[str, TestsuiteType]:
+ """
+ Read-only property returning a reference to the internal dictionary of test suites.
+
+ :return: Reference to the dictionary of test suite.
+ """
+ return self._testsuites
+
+ @readonly
+ def TestsuiteCount(self) -> int:
+ """
+ Read-only property returning the number of all test suites in the test suite hierarchy.
+
+ :return: Number of test suites.
+ """
+ return 1 + sum(testsuite.TestsuiteCount for testsuite in self._testsuites.values())
+
+ @readonly
+ def TestcaseCount(self) -> int:
+ """
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ return sum(testsuite.TestcaseCount for testsuite in self._testsuites.values())
+
+ @readonly
+ def AssertionCount(self) -> int:
+ """
+ Read-only property returning the number of all assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of assertions in all test cases.
+ """
+ return sum(ts.AssertionCount for ts in self._testsuites.values())
+
+ @readonly
+ def FailedAssertionCount(self) -> int:
+ """
+ Read-only property returning the number of all failed assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of failed assertions in all test cases.
+ """
+ raise NotImplementedError()
+ # return self._assertionCount - (self._warningCount + self._errorCount + self._fatalCount)
+
+ @readonly
+ def PassedAssertionCount(self) -> int:
+ """
+ Read-only property returning the number of all passed assertions in all test cases in the test entity hierarchy.
+
+ :return: Number of passed assertions in all test cases.
+ """
+ raise NotImplementedError()
+ # return self._assertionCount - (self._warningCount + self._errorCount + self._fatalCount)
+
+ @readonly
+ def Tests(self) -> int:
+ return self._tests
+
+ @readonly
+ def Inconsistent(self) -> int:
+ """
+ Read-only property returning the number of inconsistent tests in the test suite hierarchy.
+
+ :return: Number of inconsistent tests.
+ """
+ return self._inconsistent
+
+ @readonly
+ def Excluded(self) -> int:
+ """
+ Read-only property returning the number of excluded tests in the test suite hierarchy.
+
+ :return: Number of excluded tests.
+ """
+ return self._excluded
+
+ @readonly
+ def Skipped(self) -> int:
+ """
+ Read-only property returning the number of skipped tests in the test suite hierarchy.
+
+ :return: Number of skipped tests.
+ """
+ return self._skipped
+
+ @readonly
+ def Errored(self) -> int:
+ """
+ Read-only property returning the number of tests with errors in the test suite hierarchy.
+
+ :return: Number of errored tests.
+ """
+ return self._errored
+
+ @readonly
+ def Weak(self) -> int:
+ """
+ Read-only property returning the number of weak tests in the test suite hierarchy.
+
+ :return: Number of weak tests.
+ """
+ return self._weak
+
+ @readonly
+ def Failed(self) -> int:
+ """
+ Read-only property returning the number of failed tests in the test suite hierarchy.
+
+ :return: Number of failed tests.
+ """
+ return self._failed
+
+ @readonly
+ def Passed(self) -> int:
+ """
+ Read-only property returning the number of passed tests in the test suite hierarchy.
+
+ :return: Number of passed tests.
+ """
+ return self._passed
+
+ @readonly
+ def WarningCount(self) -> int:
+ raise NotImplementedError()
+ # return self._warningCount
+
+ @readonly
+ def ErrorCount(self) -> int:
+ raise NotImplementedError()
+ # return self._errorCount
+
+ @readonly
+ def FatalCount(self) -> int:
+ raise NotImplementedError()
+ # return self._fatalCount
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests = 0
+ inconsistent = 0
+ excluded = 0
+ skipped = 0
+ errored = 0
+ weak = 0
+ failed = 0
+ passed = 0
+
+ warningCount = 0
+ errorCount = 0
+ fatalCount = 0
+
+ totalDuration = timedelta()
+
+ for testsuite in self._testsuites.values():
+ t, i, ex, s, e, w, f, p, wc, ec, fc, td = testsuite.Aggregate(strict)
+ tests += t
+ inconsistent += i
+ excluded += ex
+ skipped += s
+ errored += e
+ weak += w
+ failed += f
+ passed += p
+
+ warningCount += wc
+ errorCount += ec
+ fatalCount += fc
+
+ totalDuration += td
+
+ return tests, inconsistent, excluded, skipped, errored, weak, failed, passed, warningCount, errorCount, fatalCount, totalDuration
+
+ def AddTestsuite(self, testsuite: TestsuiteType) -> None:
+ """
+ Add a test suite to the list of test suites.
+
+ :param testsuite: The test suite to add.
+ :raises ValueError: If parameter 'testsuite' is None.
+ :raises TypeError: If parameter 'testsuite' is not a Testsuite.
+ :raises AlreadyInHierarchyException: If parameter 'testsuite' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If parameter 'testsuite' is already listed (by name) in the list of test suites.
+ """
+ if testsuite is None:
+ raise ValueError("Parameter 'testsuite' is None.")
+ elif not isinstance(testsuite, Testsuite):
+ ex = TypeError(f"Parameter 'testsuite' is not of type 'Testsuite'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuite)}'.")
+ raise ex
+
+ if testsuite._parent is not None:
+ raise AlreadyInHierarchyException(f"Testsuite '{testsuite._name}' is already part of a testsuite hierarchy.")
+
+ if testsuite._name in self._testsuites:
+ raise DuplicateTestsuiteException(f"Testsuite already contains a testsuite with same name '{testsuite._name}'.")
+
+ testsuite._parent = self
+ self._testsuites[testsuite._name] = testsuite
+
+ def AddTestsuites(self, testsuites: Iterable[TestsuiteType]) -> None:
+ """
+ Add a list of test suites to the list of test suites.
+
+ :param testsuites: List of test suites to add.
+ :raises ValueError: If parameter 'testsuites' is None.
+ :raises TypeError: If parameter 'testsuites' is not iterable.
+ """
+ if testsuites is None:
+ raise ValueError("Parameter 'testsuites' is None.")
+ elif not isinstance(testsuites, Iterable):
+ ex = TypeError(f"Parameter 'testsuites' is not iterable.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testsuites)}'.")
+ raise ex
+
+ for testsuite in testsuites:
+ self.AddTestsuite(testsuite)
+
+ @abstractmethod
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[TestsuiteType, Testcase], None, None]:
+ pass
+
+ def IterateTestsuites(self, scheme: IterationScheme = IterationScheme.TestsuiteDefault) -> Generator[TestsuiteType, None, None]:
+ return self.Iterate(scheme)
+
+ def IterateTestcases(self, scheme: IterationScheme = IterationScheme.TestcaseDefault) -> Generator[Testcase, None, None]:
+ return self.Iterate(scheme)
+
+ def ToTree(self) -> Node:
+ rootNode = Node(value=self._name)
+
+ def convertTestcase(testcase: Testcase, parentNode: Node) -> None:
+ _ = Node(value=testcase._name, parent=parentNode)
+
+ def convertTestsuite(testsuite: Testsuite, parentNode: Node) -> None:
+ testsuiteNode = Node(value=testsuite._name, parent=parentNode)
+
+ for ts in testsuite._testsuites.values():
+ convertTestsuite(ts, testsuiteNode)
+
+ for tc in testsuite._testcases.values():
+ convertTestcase(tc, testsuiteNode)
+
+ for testsuite in self._testsuites.values():
+ convertTestsuite(testsuite, rootNode)
+
+ return rootNode
+
+
+@export
+class Testsuite(TestsuiteBase[TestsuiteType]):
+ """
+ A testsuite is a mid-level element in the test entity hierarchy representing a group of tests.
+
+ Test suites contain test cases and optionally other test suites. Test suites can be grouped by test suites to form a
+ hierarchy of test entities. The root of the hierarchy is a test summary.
+ """
+
+ _testcases: Dict[str, "Testcase"]
+
+ def __init__(
+ self,
+ name: str,
+ kind: TestsuiteKind = TestsuiteKind.Logical,
+ startTime: Nullable[datetime] = None,
+ setupDuration: Nullable[timedelta] = None,
+ testDuration: Nullable[timedelta] = None,
+ teardownDuration: Nullable[timedelta] = None,
+ totalDuration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ warningCount: int = 0,
+ errorCount: int = 0,
+ fatalCount: int = 0,
+ testsuites: Nullable[Iterable[TestsuiteType]] = None,
+ testcases: Nullable[Iterable["Testcase"]] = None,
+ keyValuePairs: Nullable[Mapping[str, Any]] = None,
+ parent: Nullable[TestsuiteType] = None
+ ):
+ """
+ Initializes the fields of a test suite.
+
+ :param name: Name of the test suite.
+ :param kind: Kind of the test suite.
+ :param startTime: Time when the test suite was started.
+ :param setupDuration: Duration it took to set up the test suite.
+ :param testDuration: Duration of all tests listed in the test suite.
+ :param teardownDuration: Duration it took to tear down the test suite.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test suite.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test suite with.
+ :param testcases: List of test cases to initialize the test suite with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test suite with.
+ :param parent: Reference to the parent test entity.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ :raises TypeError: If element in parameter 'testcases' is not a Testcase.
+ :raises AlreadyInHierarchyException: If a test case in parameter 'testcases' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If a test case in parameter 'testcases' is already listed (by name) in the list of test cases.
+ """
+ super().__init__(
+ name,
+ kind,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ status,
+ warningCount,
+ errorCount,
+ fatalCount,
+ testsuites,
+ keyValuePairs,
+ parent
+ )
+
+ # self._testDuration = testDuration
+
+ self._testcases = {}
+ if testcases is not None:
+ if not isinstance(testcases, Iterable):
+ ex = TypeError(f"Parameter 'testcases' is not iterable.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcases)}'.")
+ raise ex
+
+ for testcase in testcases:
+ if not isinstance(testcase, Testcase):
+ ex = TypeError(f"Element of parameter 'testcases' is not of type 'Testcase'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcase)}'.")
+ raise ex
+
+ if testcase._parent is not None:
+ raise AlreadyInHierarchyException(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ if testcase._name in self._testcases:
+ raise DuplicateTestcaseException(f"Testsuite already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent = self
+ self._testcases[testcase._name] = testcase
+
+ @readonly
+ def Testcases(self) -> Dict[str, "Testcase"]:
+ """
+ Read-only property returning a reference to the internal dictionary of test cases.
+
+ :return: Reference to the dictionary of test cases.
+ """
+ return self._testcases
+
+ @readonly
+ def TestcaseCount(self) -> int:
+ """
+ Read-only property returning the number of all test cases in the test entity hierarchy.
+
+ :return: Number of test cases.
+ """
+ return super().TestcaseCount + len(self._testcases)
+
+ @readonly
+ def AssertionCount(self) -> int:
+ return super().AssertionCount + sum(tc.AssertionCount for tc in self._testcases.values())
+
+ def Copy(self) -> "Testsuite":
+ return self.__class__(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount
+ )
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, inconsistent, excluded, skipped, errored, weak, failed, passed, warningCount, errorCount, fatalCount, totalDuration = super().Aggregate()
+
+ for testcase in self._testcases.values():
+ wc, ec, fc, td = testcase.Aggregate(strict)
+
+ tests += 1
+
+ warningCount += wc
+ errorCount += ec
+ fatalCount += fc
+
+ totalDuration += td
+
+ status = testcase._status
+ if status is TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testcase '{testcase._name}' with state 'Unknown'.")
+ elif TestcaseStatus.Inconsistent in status:
+ inconsistent += 1
+ elif status is TestcaseStatus.Excluded:
+ excluded += 1
+ elif status is TestcaseStatus.Skipped:
+ skipped += 1
+ elif status is TestcaseStatus.Errored:
+ errored += 1
+ elif status is TestcaseStatus.Weak:
+ weak += 1
+ elif status is TestcaseStatus.Passed:
+ passed += 1
+ elif status is TestcaseStatus.Failed:
+ failed += 1
+ elif status & TestcaseStatus.Mask is not TestcaseStatus.Unknown:
+ raise UnittestException(f"Found testcase '{testcase._name}' with unsupported state '{status}'.")
+ else:
+ raise UnittestException(f"Internal error for testcase '{testcase._name}', field '_status' is '{status}'.")
+
+ self._tests = tests
+ self._inconsistent = inconsistent
+ self._excluded = excluded
+ self._skipped = skipped
+ self._errored = errored
+ self._weak = weak
+ self._failed = failed
+ self._passed = passed
+
+ self._warningCount = warningCount
+ self._errorCount = errorCount
+ self._fatalCount = fatalCount
+
+ if self._totalDuration is None:
+ self._totalDuration = totalDuration
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, inconsistent, excluded, skipped, errored, weak, failed, passed, warningCount, errorCount, fatalCount, totalDuration
+
+ def AddTestcase(self, testcase: "Testcase") -> None:
+ """
+ Add a test case to the list of test cases.
+
+ :param testcase: The test case to add.
+ :raises ValueError: If parameter 'testcase' is None.
+ :raises TypeError: If parameter 'testcase' is not a Testcase.
+ :raises AlreadyInHierarchyException: If parameter 'testcase' is already part of a test entity hierarchy.
+ :raises DuplicateTestcaseException: If parameter 'testcase' is already listed (by name) in the list of test cases.
+ """
+ if testcase is None:
+ raise ValueError("Parameter 'testcase' is None.")
+ elif not isinstance(testcase, Testcase):
+ ex = TypeError(f"Parameter 'testcase' is not of type 'Testcase'.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcase)}'.")
+ raise ex
+
+ if testcase._parent is not None:
+ raise ValueError(f"Testcase '{testcase._name}' is already part of a testsuite hierarchy.")
+
+ if testcase._name in self._testcases:
+ raise DuplicateTestcaseException(f"Testsuite already contains a testcase with same name '{testcase._name}'.")
+
+ testcase._parent = self
+ self._testcases[testcase._name] = testcase
+
+ def AddTestcases(self, testcases: Iterable["Testcase"]) -> None:
+ """
+ Add a list of test cases to the list of test cases.
+
+ :param testcases: List of test cases to add.
+ :raises ValueError: If parameter 'testcases' is None.
+ :raises TypeError: If parameter 'testcases' is not iterable.
+ """
+ if testcases is None:
+ raise ValueError("Parameter 'testcases' is None.")
+ elif not isinstance(testcases, Iterable):
+ ex = TypeError(f"Parameter 'testcases' is not iterable.")
+ if version_info >= (3, 11): # pragma: no cover
+ ex.add_note(f"Got type '{getFullyQualifiedName(testcases)}'.")
+ raise ex
+
+ for testcase in testcases:
+ self.AddTestcase(testcase)
+
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[TestsuiteType, Testcase], None, None]:
+ assert IterationScheme.PreOrder | IterationScheme.PostOrder not in scheme
+
+ if IterationScheme.PreOrder in scheme:
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites in scheme:
+ yield self
+
+ if IterationScheme.IncludeTestcases in scheme:
+ for testcase in self._testcases.values():
+ yield testcase
+
+ for testsuite in self._testsuites.values():
+ yield from testsuite.Iterate(scheme | IterationScheme.IncludeSelf)
+
+ if IterationScheme.PostOrder in scheme:
+ if IterationScheme.IncludeTestcases in scheme:
+ for testcase in self._testcases.values():
+ yield testcase
+
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites in scheme:
+ yield self
+
+ def __str__(self) -> str:
+ return (
+ f"<Testsuite {self._name}: {self._status.name} -"
+ # f" assert/pass/fail:{self._assertionCount}/{self._passedAssertionCount}/{self._failedAssertionCount} -"
+ f" warn/error/fatal:{self._warningCount}/{self._errorCount}/{self._fatalCount}>"
+ )
+
+
+@export
+class TestsuiteSummary(TestsuiteBase[TestsuiteType]):
+ """
+ A testsuite summary is the root element in the test entity hierarchy representing a summary of all test suites and cases.
+
+ The testsuite summary contains test suites, which in turn can contain test suites and test cases.
+ """
+
+ def __init__(
+ self,
+ name: str,
+ startTime: Nullable[datetime] = None,
+ setupDuration: Nullable[timedelta] = None,
+ testDuration: Nullable[timedelta] = None,
+ teardownDuration: Nullable[timedelta] = None,
+ totalDuration: Nullable[timedelta] = None,
+ status: TestsuiteStatus = TestsuiteStatus.Unknown,
+ warningCount: int = 0,
+ errorCount: int = 0,
+ fatalCount: int = 0,
+ testsuites: Nullable[Iterable[TestsuiteType]] = None,
+ keyValuePairs: Nullable[Mapping[str, Any]] = None,
+ parent: Nullable[TestsuiteType] = None
+ ):
+ """
+ Initializes the fields of a test summary.
+
+ :param name: Name of the test summary.
+ :param startTime: Time when the test summary was started.
+ :param setupDuration: Duration it took to set up the test summary.
+ :param testDuration: Duration of all tests listed in the test summary.
+ :param teardownDuration: Duration it took to tear down the test summary.
+ :param totalDuration: Total duration of the entity's execution (setup + test + teardown)
+ :param status: Overall status of the test summary.
+ :param warningCount: Count of encountered warnings incl. warnings from sub-elements.
+ :param errorCount: Count of encountered errors incl. errors from sub-elements.
+ :param fatalCount: Count of encountered fatal errors incl. fatal errors from sub-elements.
+ :param testsuites: List of test suites to initialize the test summary with.
+ :param keyValuePairs: Mapping of key-value pairs to initialize the test summary with.
+ :param parent: Reference to the parent test summary.
+ """
+ super().__init__(
+ name,
+ TestsuiteKind.Root,
+ startTime,
+ setupDuration,
+ testDuration,
+ teardownDuration,
+ totalDuration,
+ status,
+ warningCount,
+ errorCount,
+ fatalCount,
+ testsuites,
+ keyValuePairs,
+ parent
+ )
+
+ def Aggregate(self, strict: bool = True) -> TestsuiteAggregateReturnType:
+ tests, inconsistent, excluded, skipped, errored, weak, failed, passed, warningCount, errorCount, fatalCount, totalDuration = super().Aggregate(strict)
+
+ self._tests = tests
+ self._inconsistent = inconsistent
+ self._excluded = excluded
+ self._skipped = skipped
+ self._errored = errored
+ self._weak = weak
+ self._failed = failed
+ self._passed = passed
+
+ self._warningCount = warningCount
+ self._errorCount = errorCount
+ self._fatalCount = fatalCount
+
+ if self._totalDuration is None:
+ self._totalDuration = totalDuration
+
+ if errored > 0:
+ self._status = TestsuiteStatus.Errored
+ elif failed > 0:
+ self._status = TestsuiteStatus.Failed
+ elif tests == 0:
+ self._status = TestsuiteStatus.Empty
+ elif tests - skipped == passed:
+ self._status = TestsuiteStatus.Passed
+ elif tests == skipped:
+ self._status = TestsuiteStatus.Skipped
+ elif tests == excluded:
+ self._status = TestsuiteStatus.Excluded
+ else:
+ self._status = TestsuiteStatus.Unknown
+
+ return tests, inconsistent, excluded, skipped, errored, weak, failed, passed, warningCount, errorCount, fatalCount, totalDuration
+
+ def Iterate(self, scheme: IterationScheme = IterationScheme.Default) -> Generator[Union[TestsuiteType, Testcase], None, None]:
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites | IterationScheme.PreOrder in scheme:
+ yield self
+
+ for testsuite in self._testsuites.values():
+ yield from testsuite.IterateTestsuites(scheme | IterationScheme.IncludeSelf)
+
+ if IterationScheme.IncludeSelf | IterationScheme.IncludeTestsuites | IterationScheme.PostOrder in scheme:
+ yield self
+
+ def __str__(self) -> str:
+ return (
+ f"<TestsuiteSummary {self._name}: {self._status.name} -"
+ # f" assert/pass/fail:{self._assertionCount}/{self._passedAssertionCount}/{self._failedAssertionCount} -"
+ f" warn/error/fatal:{self._warningCount}/{self._errorCount}/{self._fatalCount}>"
+ )
+
+
+@export
+class Document(metaclass=ExtendedType, mixin=True):
+ """A mixin-class representing a unit test summary document (file)."""
+
+ _path: Path
+
+ _analysisDuration: float #: TODO: replace by Timer; should be timedelta?
+ _modelConversion: float #: TODO: replace by Timer; should be timedelta?
+
+ def __init__(self, reportFile: Path, analyzeAndConvert: bool = False):
+ self._path = reportFile
+
+ self._analysisDuration = -1.0
+ self._modelConversion = -1.0
+
+ if analyzeAndConvert:
+ self.Analyze()
+ self.Convert()
+
+ @readonly
+ def Path(self) -> Path:
+ """
+ Read-only property returning the path to the file of this document.
+
+ :return: The document's path to the file.
+ """
+ return self._path
+
+ @readonly
+ def AnalysisDuration(self) -> timedelta:
+ """
+ Read-only property returning analysis duration.
+
+ .. note::
+
+ This includes usually the duration to validate and parse the file format, but it excludes the time to convert the
+ content to the test entity hierarchy.
+
+ :return: Duration to analyze the document.
+ """
+ return timedelta(seconds=self._analysisDuration)
+
+ @readonly
+ def ModelConversionDuration(self) -> timedelta:
+ """
+ Read-only property returning conversion duration.
+
+ .. note::
+
+ This includes usually the duration to convert the document's content to the test entity hierarchy. It might also
+ include the duration to (re-)aggregate all states and statistics in the hierarchy.
+
+ :return: Duration to convert the document.
+ """
+ return timedelta(seconds=self._modelConversion)
+
+ @abstractmethod
+ def Analyze(self) -> None:
+ """Analyze and validate the document's content."""
+
+ # @abstractmethod
+ # def Write(self, path: Nullable[Path] = None, overwrite: bool = False):
+ # pass
+
+ @abstractmethod
+ def Convert(self):
+ """Convert the document's content to an instance of the test entity hierarchy."""
+
+
+@export
+class Merged(metaclass=ExtendedType, mixin=True):
+ """A mixin-class representing a merged test entity."""
+
+ _mergedCount: int
+
+ def __init__(self, mergedCount: int = 1):
+ self._mergedCount = mergedCount
+
+ @readonly
+ def MergedCount(self) -> int:
+ return self._mergedCount
+
+
+@export
+class Combined(metaclass=ExtendedType, mixin=True):
+ _combinedCount: int
+
+ def __init__(self, combinedCound: int = 1):
+ self._combinedCount = combinedCound
+
+ @readonly
+ def CombinedCount(self) -> int:
+ return self._combinedCount
+
+
+@export
+class MergedTestcase(Testcase, Merged):
+ _mergedTestcases: List[Testcase]
+
+ def __init__(
+ self,
+ testcase: Testcase,
+ parent: Nullable["Testsuite"] = None
+ ):
+ if testcase is None:
+ raise ValueError(f"Parameter 'testcase' is None.")
+
+ super().__init__(
+ testcase._name,
+ testcase._startTime,
+ testcase._setupDuration, testcase._testDuration, testcase._teardownDuration, testcase._totalDuration,
+ TestcaseStatus.Unknown,
+ testcase._assertionCount, testcase._failedAssertionCount, testcase._passedAssertionCount,
+ testcase._warningCount, testcase._errorCount, testcase._fatalCount,
+ parent
+ )
+ Merged.__init__(self)
+
+ self._mergedTestcases = [testcase]
+
+ @readonly
+ def Status(self) -> TestcaseStatus:
+ if self._status is TestcaseStatus.Unknown:
+ status = self._mergedTestcases[0]._status
+ for mtc in self._mergedTestcases[1:]:
+ status @= mtc._status
+
+ self._status = status
+
+ return self._status
+
+ @readonly
+ def SummedAssertionCount(self) -> int:
+ return sum(tc._assertionCount for tc in self._mergedTestcases)
+
+ @readonly
+ def SummedPassedAssertionCount(self) -> int:
+ return sum(tc._passedAssertionCount for tc in self._mergedTestcases)
+
+ @readonly
+ def SummedFailedAssertionCount(self) -> int:
+ return sum(tc._failedAssertionCount for tc in self._mergedTestcases)
+
+ def Aggregate(self, strict: bool = True) -> TestcaseAggregateReturnType:
+ firstMTC = self._mergedTestcases[0]
+
+ status = firstMTC._status
+ warningCount = firstMTC._warningCount
+ errorCount = firstMTC._errorCount
+ fatalCount = firstMTC._fatalCount
+ totalDuration = firstMTC._totalDuration
+
+ for mtc in self._mergedTestcases[1:]:
+ status @= mtc._status
+ warningCount += mtc._warningCount
+ errorCount += mtc._errorCount
+ fatalCount += mtc._fatalCount
+
+ self._status = status
+
+ return warningCount, errorCount, fatalCount, totalDuration
+
+ def Merge(self, tc: Testcase) -> None:
+ self._mergedCount += 1
+
+ self._mergedTestcases.append(tc)
+
+ self._warningCount += tc._warningCount
+ self._errorCount += tc._errorCount
+ self._fatalCount += tc._fatalCount
+
+ def ToTestcase(self) -> Testcase:
+ return Testcase(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._testDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._assertionCount,
+ self._failedAssertionCount,
+ self._passedAssertionCount,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount
+ )
+
+
+@export
+class MergedTestsuite(Testsuite, Merged):
+ def __init__(
+ self,
+ testsuite: Testsuite,
+ addTestsuites: bool = False,
+ addTestcases: bool = False,
+ parent: Nullable["Testsuite"] = None
+ ):
+ if testsuite is None:
+ raise ValueError(f"Parameter 'testsuite' is None.")
+
+ super().__init__(
+ testsuite._name,
+ testsuite._kind,
+ testsuite._startTime,
+ testsuite._setupDuration, testsuite._testDuration, testsuite._teardownDuration, testsuite._totalDuration,
+ TestsuiteStatus.Unknown,
+ testsuite._warningCount, testsuite._errorCount, testsuite._fatalCount,
+ parent
+ )
+ Merged.__init__(self)
+
+ if addTestsuites:
+ for ts in testsuite._testsuites.values():
+ mergedTestsuite = MergedTestsuite(ts, addTestsuites, addTestcases)
+ self.AddTestsuite(mergedTestsuite)
+
+ if addTestcases:
+ for tc in testsuite._testcases.values():
+ mergedTestcase = MergedTestcase(tc)
+ self.AddTestcase(mergedTestcase)
+
+ def Merge(self, testsuite: Testsuite) -> None:
+ self._mergedCount += 1
+
+ for ts in testsuite._testsuites.values():
+ if ts._name in self._testsuites:
+ self._testsuites[ts._name].Merge(ts)
+ else:
+ mergedTestsuite = MergedTestsuite(ts, addTestsuites=True, addTestcases=True)
+ self.AddTestsuite(mergedTestsuite)
+
+ for tc in testsuite._testcases.values():
+ if tc._name in self._testcases:
+ self._testcases[tc._name].Merge(tc)
+ else:
+ mergedTestcase = MergedTestcase(tc)
+ self.AddTestcase(mergedTestcase)
+
+ def ToTestsuite(self) -> Testsuite:
+ testsuite = Testsuite(
+ self._name,
+ self._kind,
+ self._startTime,
+ self._setupDuration,
+ self._testDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount,
+ testsuites=(ts.ToTestsuite() for ts in self._testsuites.values()),
+ testcases=(tc.ToTestcase() for tc in self._testcases.values())
+ )
+
+ testsuite._tests = self._tests
+ testsuite._excluded = self._excluded
+ testsuite._inconsistent = self._inconsistent
+ testsuite._skipped = self._skipped
+ testsuite._errored = self._errored
+ testsuite._weak = self._weak
+ testsuite._failed = self._failed
+ testsuite._passed = self._passed
+
+ return testsuite
+
+
+@export
+class MergedTestsuiteSummary(TestsuiteSummary, Merged):
+ _mergedFiles: Dict[Path, TestsuiteSummary]
+
+ def __init__(self, name: str) -> None:
+ super().__init__(name)
+ Merged.__init__(self, mergedCount=0)
+
+ self._mergedFiles = {}
+
+ def Merge(self, testsuiteSummary: TestsuiteSummary) -> None:
+ # if summary.File in self._mergedFiles:
+ # raise
+
+ # FIXME: a summary is not necessarily a file
+ self._mergedCount += 1
+ self._mergedFiles[testsuiteSummary._name] = testsuiteSummary
+
+ for testsuite in testsuiteSummary._testsuites.values():
+ if testsuite._name in self._testsuites:
+ self._testsuites[testsuite._name].Merge(testsuite)
+ else:
+ mergedTestsuite = MergedTestsuite(testsuite, addTestsuites=True, addTestcases=True)
+ self.AddTestsuite(mergedTestsuite)
+
+ def ToTestsuiteSummary(self) -> TestsuiteSummary:
+ testsuiteSummary = TestsuiteSummary(
+ self._name,
+ self._startTime,
+ self._setupDuration,
+ self._testDuration,
+ self._teardownDuration,
+ self._totalDuration,
+ self._status,
+ self._warningCount,
+ self._errorCount,
+ self._fatalCount,
+ testsuites=(ts.ToTestsuite() for ts in self._testsuites.values())
+ )
+
+ testsuiteSummary._tests = self._tests
+ testsuiteSummary._excluded = self._excluded
+ testsuiteSummary._inconsistent = self._inconsistent
+ testsuiteSummary._skipped = self._skipped
+ testsuiteSummary._errored = self._errored
+ testsuiteSummary._weak = self._weak
+ testsuiteSummary._failed = self._failed
+ testsuiteSummary._passed = self._passed
+
+ return testsuiteSummary
+
+
+