Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for diagnostic messages on stderr, not stdout #1540

Merged

Conversation

evan-goode
Copy link
Member

For rpm-software-management/dnf5#1641.

Adds support for the <REPOSYNC> shorthand to Then stderr is and Then stderr matches line by line.

Introduces Then stdout contains lines matching and Then stderr contains lines matching steps which are similar to Then stdout contains lines but treat each expected line as a regex, like Then stdout matches line by line.

This patchset strengthens many tests; there were many instances of the pattern:

Then stdout is
     """
     <REPOSYNC>
     foo
     bar
     """

that were replaced with

Then stderr is
     """
     <REPOSYNC>
     """
 And stdout is
     """
     foo
     bar
     """

so now stderr is checked in many places where it wasn't before.

However, some tests were weakened, notably those following the pattern:

Then stderr is
     """
     No match for argument: nosuchpkg

     Warning: skipped PGP checks for 1 package from repository: dnf-ci-fedora-updates
     """

Because stderr now includes lots of other (non-error) information including progress bars for downloads and RPM transactions, these occurrences were changed to:

Then stderr contains lines
     """
     No match for argument: nosuchpkg
     Warning: skipped PGP checks for 1 package from repository: dnf-ci-fedora-updates
     """

We could maybe make <RPMDOWNLOAD> and <RPMTRANSACTION> markers that work similar to <REPOSYNC> to allow us to use Then stderr is again, but I'd rather do that in a follow-up issue.

These changes were made mostly manually with the help of Neovim macros.

@jan-kolarik
Copy link
Member

/packit test rpm-software-management/dnf5#1641

Copy link
Member

@jan-kolarik jan-kolarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tests were run locally successfully. And thanks for adding some docs to the python functions here in the CI stack!

@jan-kolarik jan-kolarik merged commit edec323 into rpm-software-management:main Sep 2, 2024
4 of 9 checks passed
evan-goode added a commit to evan-goode/ci-dnf-stack that referenced this pull request Sep 5, 2024
After merging rpm-software-management#1540, it was revealed that this test in module/updateinfo.feature wasn't actually testing anything, and the behavior it intends to test is (and has long been?) broken. The test was expecting an empty stdout (or <REPOSYNC>), which it was getting, but only after refactoring the tests did we notice the message on stderr:

Unknown argument "--list" for command "updateinfo".

For now, I've updated the test to call the correct command, `updateinfo
list`, and disabled the test since it is now failing (but in a more
valid way).
jan-kolarik pushed a commit that referenced this pull request Sep 6, 2024
After merging #1540, it was revealed that this test in module/updateinfo.feature wasn't actually testing anything, and the behavior it intends to test is (and has long been?) broken. The test was expecting an empty stdout (or <REPOSYNC>), which it was getting, but only after refactoring the tests did we notice the message on stderr:

Unknown argument "--list" for command "updateinfo".

For now, I've updated the test to call the correct command, `updateinfo
list`, and disabled the test since it is now failing (but in a more
valid way).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants