Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added option to make deferred evaluation quieter #1171

Closed
wants to merge 9 commits into from

Conversation

epinzur
Copy link
Contributor

@epinzur epinzur commented Jun 3, 2024

Items to add to release announcement:

  • Added Option to make deferred evaluation quieter:
    • If using Deferred evaluation mode with the tru.start_evaluator() method, there is a new disable_tqdm flag (default False), that disables the tqdm progress bars written to stdout when set True.
    • This is useful when you are managing the evaluation progress via some other method.

Other details that are good to know but need not be announced:

  • This also disables:
    • the nltk.download() logging that appears very often in the deferred evaluation log output.
    • the progress bars from the groundedness_measure_with_cot_reasons feedback function.
      • Note: I wanted to make a flag to also optionally disable these progress bars, but I was un-successful in doing so.

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jun 3, 2024
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Jun 6, 2024
@epinzur epinzur closed this Jun 6, 2024
@epinzur epinzur deleted the esp/quieter_eval branch June 6, 2024 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant