Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: pass test if files have holes; don't write out file if checking fails #44

Merged
merged 8 commits into from
Oct 26, 2024

Conversation

acl-cqc
Copy link
Collaborator

@acl-cqc acl-cqc commented Oct 25, 2024

  • First, refactor: move CompilingHoles out of Error; it is only used in one place, where there is no other possible Error. Make CompileFile return CompilingHoles rather than String.
  • Second, pass compilation rather than (expected-)fail if there are holes. This does seem a bit silent, so other options (all a bit harder):
    • Maintain a separate list of files with holes that we expect to fail in this way
    • Do checking first, and only generate a compilation test for files that check without any holes. That way, these compilation tests disappear from the list of run tests, rather than appearing to pass.
  • Third, fix the weird bug that files that errord in compilation, still returned an empty bytestring. This is not what I'd have expected from error but the fix seems to be to use evaluate from Control.Exception getting us an IO ByteString that dies if there was an error.

@acl-cqc acl-cqc requested a review from croyzor October 25, 2024 14:30
Copy link
Collaborator

@croyzor croyzor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice if we could have a pytest-style "(Skipped)" reporting to be a little bit less silent, but it's not immediately obvious to me how to achive this using tasty

@acl-cqc
Copy link
Collaborator Author

acl-cqc commented Oct 25, 2024

It'd be nice if we could have a pytest-style "(Skipped)" reporting to be a little bit less silent, but it's not immediately obvious to me how to achive this using tasty

testCaseInfo rather than testCase sort of does it - we have to display a message for every testCase using that, which here means every compilation test, but for the passing ones we can use that to hint that there's another step still to do (i.e. validation).

TBH I think a better approach still would be to use testCaseSteps to combine parsing, checking and compilation into a single multi-step test, but that's well beyond the scope of this PR ;)

@acl-cqc acl-cqc merged commit 96cb8bc into main Oct 26, 2024
1 check passed
@acl-cqc acl-cqc deleted the tests/compilation_holes_nonempty branch October 26, 2024 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants