Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

total count of gold matches in test mode #28

Open
jonorthwash opened this issue Feb 27, 2023 · 3 comments
Open

total count of gold matches in test mode #28

jonorthwash opened this issue Feb 27, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@jonorthwash
Copy link
Member

Currently apertium-regtest test has output like this:

$ apertium-regtest -c .-morph test
Corpus 1 of 3: -s plurals-morph
  13/13 (100.0%) tests pass (3/13 (23.08%) match gold)

Corpus 2 of 3: -es plurals-morph
  20/20 (100.0%) tests pass (3/20 (15.0%) match gold)

Corpus 3 of 3: irregular plurals-morph
  22/22 (100.0%) tests pass (2/22 (9.09%) match gold)

All tests pass.

It would be nice if the bottom line (or somewhere around there) said something like "8/55 tests match gold" or similar.

@jonorthwash jonorthwash added the enhancement New feature or request label Feb 27, 2023
@jonorthwash
Copy link
Member Author

Also, the wording "all tests pass" is misleading when so many don't match gold. Perhaps that wording should be clarified too.

@TinoDidriksen
Copy link
Member

"All tests pass" is correct. Gold is not relevant to test passing - gold is just an extra step to keep track of "perfect" translations, but has nothing to do with whether the translation changed.
This is not a gold tester, and must not be a gold tester. It's an important distinction and different mindset.

@jonorthwash
Copy link
Member Author

This is not a gold tester, and must not be a gold tester.

Okay, sure, but for morphology this will be the primary use, especially for those of us using it as a replacement for aq-morphtest and morhp-test.py. So, okay, leave the terminology and mindset as is, but we could still use some minor additional features that resemble gold testing to make it more useful for morphology work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants