Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for Expecting an Error #21

Open
evan-gordon opened this issue May 18, 2024 · 1 comment
Open

Add Support for Expecting an Error #21

evan-gordon opened this issue May 18, 2024 · 1 comment

Comments

@evan-gordon
Copy link

We would like to be able to formally expect errors in tests. This would help formalizing cases where an engine should fail to execute. Today there are tests with comment tags that say something to the effect of "this should be an error", making this change would formalize these cases.

Option One:

Add an tag for tests where we expect the test to return an error

Option Two:

Place tests that we expect to return error in a different location.

Out of scope:

We could have a discussion on formalizing error representations in CQL engines but for the purposes of this issue we should just worry about whether or not a test should return an error.

@brynrhodes
Copy link
Member

The current test specification supports this, and the runner takes advantage of this capability, see the evaluation of test results here:

https://github.com/cqframework/cql-tests-runner/blob/main/cql-tests-runner.js#L65

and here:

https://github.com/cqframework/cql-tests-runner/blob/main/cql-tests-runner.js#L185

Note that one of the topics we discussed at the WGM was creating error codes for the errors so that we can test error codes, rather than testing that specific messages are coming out, but that's a different issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants