Skip to content

Commit

Permalink
Fixed scorer docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mattpocock committed Dec 18, 2024
1 parent 40c3be7 commit bb16cd5
Showing 1 changed file with 21 additions and 2 deletions.
23 changes: 21 additions & 2 deletions apps/evalite-docs/src/content/docs/guides/scorers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ import { createScorer } from "evalite";
const containsParis = createScorer<string>({
name: "Contains Paris",
description: "Checks if the output contains the word 'Paris'.",
score: (output) => {
score: ({ output }) => {
return output.includes("Paris") ? 1 : 0;
},
});

evalite("My Eval", {
data: async () => {
return [{ input: "Hello", output: "Hello World!" }];
return [{ input: "Hello" }];
},
task: async (input) => {
return input + " World!";
Expand All @@ -34,6 +34,25 @@ evalite("My Eval", {

The `name` and `description` of the scorer will be displayed in the Evalite UI.

### Score Properties

The `score` function receives three properties on the object passed:

```ts
import { createScorer } from "evalite";

const containsParis = createScorer({
name: "Contains Paris",
description: "Checks if the output contains the word 'Paris'.",
score: ({ input, output, expected }) => {
// input comes from `data`
// expected also comes from `data`
// output is the output of `task`
return output.includes("Paris") ? 1 : 0;
},
});
```

## Type Arguments

In TypeScript, `createScorer` takes two type arguments, the input and output type:
Expand Down

0 comments on commit bb16cd5

Please sign in to comment.