Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin stdout_gtestlike_tests_report does not show information about ignored tests #820

Open
i-adamov opened this issue Oct 9, 2023 · 3 comments

Comments

@i-adamov
Copy link

i-adamov commented Oct 9, 2023

The stdout_gtestlike_tests_report plugin does not have a concept of ignored tests and instead shows the same "OK" status next to them. The ignored tests are not mentioned at the summary at the end and messages printed with the TEST_IGNORE_MESSAGE are not present in this plugin's output. In the project I am working on, we use TEST_IGNORE and TEST_IGNORE_MESSAGE to mark tests for functionality that is not yet implemented and we use the gtestlike report printing because it is easier to read in our CI stage output.

Replicate by:

  1. Enable the plugin in project.yml file
:plugins:
  :enabled:
    - stdout_gtestlike_tests_report
  1. Create a test with and use in it the TEST_IGNORE or TEST_IGNORE_MESSAGE macro
  2. Run the tests - summary printout shows all test passed, including the ignored test and there is no mention of ignoring anything.
...
[----------] Global test environment tear-down.
[==========] 577 tests from 0 test cases ran.
[  PASSED  ] 488 tests.
[  FAILED  ] 0 tests.

you can see that 89 tests were ignored by looking at the difference between 577 tests and 488 passed but there is no other information available to see which were ignored and their ignore messages

@mvandervoord
Copy link
Member

At the time we wrote this, gtest did not support ignored tests. Has this changed and they have a standard way of displaying ignored tests? If not, we can update the plugin to also display this. Otherwise, the point of the gtest plugin was to output exactly like gtest so that parsers designed for gtest would understand it.

@i-adamov
Copy link
Author

i-adamov commented Oct 9, 2023

We are using gtestlike reports because they look good. I understand now that its not a problem of ceedling but of gtest that it has no matching concept of ignored tests. I guess this issue is not an issue and can be closed

Thank you!

@mvandervoord
Copy link
Member

:) It'd probably be simple enough to make them optional, and just default to disabled. You can leave this issue open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants