Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: <Issue related to /evaluation/tutorials/testing> #653

Open
labdmitriy opened this issue Jan 28, 2025 · 0 comments
Open

DOC: <Issue related to /evaluation/tutorials/testing> #653

labdmitriy opened this issue Jan 28, 2025 · 0 comments

Comments

@labdmitriy
Copy link

Thank you for tutorial!

I had the following issues while trying to reproduce it:

  • Some variables in test 4 (like search_results and answer_and_docs are not properly formatted
  • Grade class for test 4 has the typo in the field description (frounded instead of grounded) and after correction even gpt-4o-mini is working correctly as judge
  • If I try to run pytest --output=ls . (my test file is in the same directory as the application), then I have the following output:
Collecting tests...
Running tests...


no tests ran in 32.96s

But I can see new test traces in LangSmith and executing pytest . shows that it can find and run tests successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant