Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use assert{Bool,Failure} from tasty-hunit #1065

Merged
merged 1 commit into from
Apr 25, 2022
Merged

Use assert{Bool,Failure} from tasty-hunit #1065

merged 1 commit into from
Apr 25, 2022

Conversation

andreasabel
Copy link
Member

Use assert{Bool,Failure} from tasty-hunit rather than hunit. Gives better error printing.

Before:

    Accept GHC 9.2 LANGUAGE extensions: FAIL
      Exception: HUnitFailure (Just (SrcLoc {srcLocPackage = "main", srcLocModule = "Main", srcLocFile = "tests/PackageTestMain.hs", srcLocStartLine = 118, srcLocStartCol = 34, srcLocEndLine = 118, srcLocEndCol = 45})) (Reason "Expected success, but got: \"Unknown extensions: GarbageBlargh\"")

After:

    Accept GHC 9.2 LANGUAGE extensions: FAIL
      tests/PackageTestMain.hs:117:
      Expected success, but got: "Unknown extensions: GarbageBlargh"

Background

Following the discussion at UnkindPartition/tasty#327 (comment) :

@andreasabel wrote:
There is a Show instance for HUnitFailure which gives the Haskell representation of this exception. So far so good.

I am looking for a function that pretty-prints the exception, for presentation to the user.
It should look nicer than what Show gives me:

      Exception: HUnitFailure (Just (SrcLoc {srcLocPackage = "main", srcLocModule = "Main", srcLocFile =
"tests/PackageTestMain.hs", srcLocStartLine = 127, srcLocStartCol = 3, srcLocEndLine = 127, srcLocEndCol =
17})) (Reason "Expected success but got: \"correct-package-0.1.0.0/correct-package.cabal:11:34: \\nunexpected
Unknown cabal spec version specified: 1.10123456\\nexpecting \\\".\\\", \\\"-\\\", white space, \\\"&&\\\" or \\\"||
\\\"\\n\"")

@VictorCMiraldo wrote:
I started working on this today but I couldn't reproduce the problem. Eventually I discovered that @andreasabel and I did something wrong. What we did is some variation on:

import qualified Test.HUnit as Raw
import qualified Test.Tasty.HUnit as Tasty

t1 = Tasty.testCase "Some Test" $ Raw.assertFailure "Omg"

The issue comes from the fact that Raw.HUnitAssertion is a different type altogether from Tasty.HUnitAssertion. Because Tasty.Assertion == Raw.Assertion == IO (), and failures are communicated through exceptions, we're bypassing the try on Tasty.HUnit.run and flagging the test as failed through a random exception, since that method is trying on Tasty.HUnitAssertion.

The solution is to not depend on HUnit, depend exclusively on tasty-hunit and use the functions from there. Your test failures will be printed properly then.

Following the discussion at

  UnkindPartition/tasty#327 (comment)

@andreasabel wrote:
There is a `Show` instance for `HUnitFailure` which gives the _Haskell_ representation of this exception.  So far so good.

I am looking for a function that _pretty-prints_ the exception, for presentation to the user.
It should look nicer than what `Show` gives me:
```
      Exception: HUnitFailure (Just (SrcLoc {srcLocPackage = "main", srcLocModule = "Main", srcLocFile =
"tests/PackageTestMain.hs", srcLocStartLine = 127, srcLocStartCol = 3, srcLocEndLine = 127, srcLocEndCol =
17})) (Reason "Expected success but got: \"correct-package-0.1.0.0/correct-package.cabal:11:34: \\nunexpected
Unknown cabal spec version specified: 1.10123456\\nexpecting \\\".\\\", \\\"-\\\", white space, \\\"&&\\\" or \\\"||
\\\"\\n\"")
```

@VictorCMiraldo wrote:
I started working on this today but I couldn't reproduce the problem. Eventually I discovered that @andreasabel and I did  something wrong. What we did is some variation on:

```haskell
import qualified Test.HUnit as Raw
import qualified Test.Tasty.HUnit as Tasty

t1 = Tasty.testCase "Some Test" $ Raw.assertFailure "Omg"
```

The issue comes from the fact that `Raw.HUnitAssertion` is a different type altogether from [`Tasty.HUnitAssertion`](https://github.com/UnkindPartition/tasty/blob/16289a77495eb8279c5e544886ef52503becd148/hunit/Test/Tasty/HUnit/Orig.hs#L136). Because `Tasty.Assertion == Raw.Assertion == IO ()`, and failures are communicated through exceptions, we're bypassing the `try` on [`Tasty.HUnit.run`](https://github.com/UnkindPartition/tasty/blob/master/hunit/Test/Tasty/HUnit.hs#L102) and flagging the test as failed through a random exception, since that method is `try`ing on `Tasty.HUnitAssertion`.

The solution is to *not* depend on `HUnit`, depend exclusively on `tasty-hunit` and use the functions from there. Your test failures will be printed properly then.
@andreasabel andreasabel added the re: testsuite Concerning the test suite label Apr 25, 2022
@andreasabel andreasabel self-assigned this Apr 25, 2022
@andreasabel andreasabel merged commit 4a46d16 into master Apr 25, 2022
@andreasabel andreasabel deleted the tasty-hunit branch April 25, 2022 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
re: testsuite Concerning the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant