Skip to content

Commit

Permalink
Add JUnit 5 ScorerExtension for AI model evaluation and associated …
Browse files Browse the repository at this point in the history
…library

- Implement `ScorerExtension` to inject and manage Scorer instances in tests.
- Support field and parameter injection for Scorer using `@ScorerConfiguration`.
- Add support for parameter injection of samples via `@SampleLocation` annotation.
- Provide built-in evaluation strategies:
  - `SemanticSimilarityStrategy` (cosine similarity-based evaluation).
  - `AiJudgeStrategy` (AI-powered evaluation with customizable prompts).
- Add tests for ScorerExtension:
  - Validate field and parameter injection of Scorer.
  - Test sample injection from YAML files.
  - Verify evaluation strategies and reporting.
- Document ScorerExtension:
  - Explain concepts: Scorer, Samples, Evaluation Strategies, Reports.
  - Usage examples for field/parameter injection and evaluation.
  - Guide for using built-in strategies and creating custom strategies.
  • Loading branch information
cescoffier committed Dec 18, 2024
1 parent 6ec8ea5 commit be27c81
Show file tree
Hide file tree
Showing 32 changed files with 2,764 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
** xref:prompt-generation.adoc[Prompt Generation]
** xref:guardrails.adoc[Guardrails]
** xref:response-augmenter.adoc[Response Augmenter]
** xref:testing.adoc[Testing]
* LLMs
** xref:llms.adoc[LLMs]
Expand Down
Loading

0 comments on commit be27c81

Please sign in to comment.