Unitprompt

This library is the start of a collection of useful prompts and tests to bulletproof your LLM application.

It extends popular unit testing frameworks to facilitate maintaining reliable results during LLM application development.

Currently supported: jest, python (pytest)

Installation

Installation for jest Installation for pytest

For some features that require LLM calls, set the environment variable sh OPENAI_API_KEY

Examples

const generatedResult = yourLLMProcess(); // assume it is 3
const question = "What is 1+2?"

test('should be concise answer', async () => {
    await expect(generatedResult).toBeConciseAnswerTo(question);
})

Development

Create a local symlink for the shared/prompts directory:

For jest inside /jest:

Windows

New-Item -ItemType SymbolicLink -Path .\prompts -Target ..\shared\prompts

Unix

ln -s ../shared/prompts ./prompts

For pytest inside /pytest/unitprompt:

Windows

New-Item -ItemType SymbolicLink -Path .\prompts -Target ..\..\shared\prompts

Unix

ln -s ../shared/prompts ./prompts