You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Hallucination metric score is being set to either 0.0 or 1.0. It never sets a score to a value in between of 0.0 and 1.0
To Reproduce
Steps to reproduce the behavior:
Create test cases where actual_output overlaps with context
Use hallucination metric to measure it
Run evaluation
Check hallucination score
Expected behavior
The hallucination score value is a continuous spectrum of real values <0.0; 1.0>
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
OS: [e.g. iOS] Windows 11
Browser [e.g. chrome, safari] Chrome
Version [e.g. 22] 122
Smartphone (please complete the following information):
Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]
Additional context
This problem might be connected with another one (the same test can get 0.0 in one run and 1.0 in another run, despite having the same input values)
The text was updated successfully, but these errors were encountered:
Describe the bug
Hallucination metric score is being set to either 0.0 or 1.0. It never sets a score to a value in between of 0.0 and 1.0
To Reproduce
Steps to reproduce the behavior:
actual_output
overlaps withcontext
Expected behavior
The hallucination score value is a continuous spectrum of real values <0.0; 1.0>
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
This problem might be connected with another one (the same test can get 0.0 in one run and 1.0 in another run, despite having the same input values)
The text was updated successfully, but these errors were encountered: