Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI improvement: option to display only failed tests #1235

Open
yuriykuzin opened this issue Dec 18, 2024 · 1 comment
Open

CLI improvement: option to display only failed tests #1235

yuriykuzin opened this issue Dec 18, 2024 · 1 comment

Comments

@yuriykuzin
Copy link

Is your feature request related to a problem? Please describe.

When using DeepEval in a Git pre-push hook, the output includes all test results. This extensive output makes it difficult to quickly identify failed tests, hindering the efficiency of the pre-push process.

Describe the solution you’d like

I would like DeepEval to offer an option or flag that allows displaying only failed tests in the output.

Describe alternatives you’ve considered

Develop a custom script using Python API to evaluate LLM outputs, bypassing the DeepEval CLI.

Additional context

Implementing this feature would be particularly beneficial for devs who integrate DeepEval into CI pipelines or automated Git hooks, where concise and relevant output is crucial.

@stevermeister
Copy link

I also expected it to be a built-in feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants