Skip to content

Commit

Permalink
Added streams section to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
mattpocock committed Dec 6, 2024
1 parent 9769ab8 commit dc507c5
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions packages/evalite/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,3 +239,30 @@ const result = await generateText({
> [!NOTE]
>
> `traceAISDKModel`, like `reportTrace`, is a no-op in production.
### Streams

You can handle streams in Evalite by returning a `ReadableStream` from your task. This means you can test functions like the AI SDK `streamText` function easily:

```ts
import { evalite } from "evalite";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
import { Factuality } from "autoevals";

evalite("My Eval", {
data: async () => {
return [{ input: "What is the capital of France?", expected: "Paris" }];
},
task: async (input) => {
const result = await streamText({
model: openai("your-model"),
system: `Answer the question concisely.`,
prompt: input,
});

return result.textStream;
},
scorers: [Factuality],
});
```

0 comments on commit dc507c5

Please sign in to comment.