[Ellipsis] LLMBasedFaithfulness does not work when not using LLMs other than OpenAI #70
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Issue: resolves #69
Implementation:
Step 1: Modify LLMBasedFaithfulness class in continuous_eval/metrics/generation/text/llm_based.py
In the
__call__
method ofLLMBasedFaithfulness
class, modify the instantiation ofLLMBasedContextCoverage
to pass themodel
parameter. Themodel
parameter should beself.model
if it is notNone
, otherwise it should beNone
. The modified code should look like this:Report:
Modify LLMBasedFaithfulness class in continuous_eval/metrics/generation/text/llm_based.py
Modified the call method of the LLMBasedFaithfulness class in the file continuous_eval/metrics/generation/text/llm_based.py to pass the model parameter to the LLMBasedContextCoverage class.Something look wrong?: If this Pull Request doesn't contain the expected changes, add more information to #69. Then, try again. For more information, check the documentation.