Skip to content

Commit

Permalink
Raise the expected failure count
Browse files Browse the repository at this point in the history
This should let the `test-fixtures-windows` job succeed.

As suspected (but not, as of now, explained), the five additional
failures when running the tests in `bash` and piping the output
went away with `pwsh` and no pipe.

This brings the number of failures down to 14, with the possibility
of 15 failing tests if the performance test fails (see discussion
in #1358). However, while it fails when I run it locally, that
performance test seems rarely if ever to fail on CI (anymore?).
So let's try setting the maximum number of failing tests, above
which the job will report failure, to 14 rather than 15.

So that they can be investigated later, the tests that failed with
`bash` and `|&` piping of stdout and stderr, when run on Windows on
CI with `GIX_TEST_IGNORE_ARCHIVES=1` -- which are listed in the
previous commit where the failures were corrected -- have as the
most significant reported details as follows:

    --- STDERR:              gix-credentials::credentials program::from_custom_definition::empty ---
    thread 'program::from_custom_definition::empty' panicked at gix-credentials\tests\program\from_custom_definition.rs:16:5:
    assertion `left == right` failed: not useful, but allowed, would have to be caught elsewhere
      left: "\"sh\" \"-c\" \"C:\\\\Program Files\\\\Git\\\\mingw64\\\\bin\\\\git.exe credential- \\\"$@\\\"\" \"--\" \"store\""
     right: "\"C:\\Program Files\\Git\\mingw64\\bin\\git.exe\" \"credential-\" \"store\""
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    --- STDERR:              gix-credentials::credentials program::from_custom_definition::name ---
    thread 'program::from_custom_definition::name' panicked at gix-credentials\tests\program\from_custom_definition.rs:64:5:
    assertion `left == right` failed: we detect that this can run without shell, which is also more portable on windows
      left: "\"sh\" \"-c\" \"C:\\\\Program Files\\\\Git\\\\mingw64\\\\bin\\\\git.exe credential-name \\\"$@\\\"\" \"--\" \"store\""
     right: "\"C:\\Program Files\\Git\\mingw64\\bin\\git.exe\" \"credential-name\" \"store\""
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    --- STDERR:              gix-credentials::credentials program::from_custom_definition::name_with_args ---
    thread 'program::from_custom_definition::name_with_args' panicked at gix-credentials\tests\program\from_custom_definition.rs:40:5:
    assertion `left == right` failed
      left: "\"sh\" \"-c\" \"C:\\\\Program Files\\\\Git\\\\mingw64\\\\bin\\\\git.exe credential-name --arg --bar=\\\"a b\\\" \\\"$@\\\"\" \"--\" \"store\""
     right: "\"C:\\Program Files\\Git\\mingw64\\bin\\git.exe\" \"credential-name\" \"--arg\" \"--bar=a b\" \"store\""
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    --- STDERR:              gix-credentials::credentials program::from_custom_definition::name_with_special_args ---
    thread 'program::from_custom_definition::name_with_special_args' panicked at gix-credentials\tests\program\from_custom_definition.rs:52:5:
    assertion `left == right` failed
      left: "\"sh\" \"-c\" \"C:\\\\Program Files\\\\Git\\\\mingw64\\\\bin\\\\git.exe credential-name --arg --bar=~/folder/in/home \\\"$@\\\"\" \"--\" \"store\""
     right: "\"sh\" \"-c\" \"C:\\Program Files\\Git\\mingw64\\bin\\git.exe credential-name --arg --bar=~/folder/in/home \\\"$@\\\"\" \"--\" \"store\""
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    --- STDERR:              gix-discover::discover upwards::from_dir_with_dot_dot ---
    thread 'upwards::from_dir_with_dot_dot' panicked at gix-discover\tests\discover\upwards\mod.rs:155:5:
    assertion `left == right` failed
      left: Reduced
     right: Full
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

To be clear, those failures do *not* occur in the preivous commit
and are not expected to occur in this one. Those prior failures
consist of four `gix-credentials` tests and one `gix-discover`
test. (The `gix-discover` failure resembles a problem observed
locally on a Windows Server 2022 system as an administrator with
UAC disabled, reported in #1429, and suggests that the conditions
of that failure may differ from, or be more complex than, what I
had described there.)
  • Loading branch information
EliahKagan committed Nov 7, 2024
1 parent 145ae49 commit 319e9d8
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,8 @@ jobs:
- name: Check how many tests failed
if: steps.nextest.outcome == 'failure'
env:
# FIXME: Change to 15 after verifying that the last step can fail.
# See https://github.com/GitoxideLabs/gitoxide/issues/1358.
EXPECTED_FAILURE_COUNT: 13
EXPECTED_FAILURE_COUNT: 14
run: |
[xml]$junit = Get-Content -Path 'target/nextest/with-xml/junit.xml'
if ($junit.testsuites.errors -ne 0) { exit 1 }
Expand Down

0 comments on commit 319e9d8

Please sign in to comment.