Skip to content

Commit

Permalink
feat: Add Redis prompt registry (#21)
Browse files Browse the repository at this point in the history
* add redis registry

* run redis in the CI

* split the jobs

* skip coverage on mac and win

* reformat

* fix linting

* add docs
  • Loading branch information
masci authored Oct 27, 2024
1 parent 43fe52a commit def2c31
Show file tree
Hide file tree
Showing 11 changed files with 622 additions and 149 deletions.
98 changes: 77 additions & 21 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ name: test
on:
push:
branches:
- main
- main
pull_request:

concurrency:
Expand All @@ -17,33 +17,89 @@ env:
FORCE_COLOR: "1"

jobs:
run:
name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
runs-on: ${{ matrix.os }}
linux:
name: Python ${{ matrix.python-version }} on Linux
runs-on: ubuntu-latest
services:
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.10', '3.11', '3.12']
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- if: matrix.python-version == '3.12'
name: Lint
run: hatch run lint:all

- name: Run tests
run: hatch run cov

- if: matrix.python-version == '3.12'
name: Report Coveralls
uses: coverallsapp/github-action@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
macos:
name: Python ${{ matrix.python-version }} on macOS
runs-on: macos-latest

- name: Install Hatch
run: pip install --upgrade hatch
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- name: Run tests
run: hatch run test

windows:
name: Python ${{ matrix.python-version }} on Windows
runs-on: windows-latest

strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- if: matrix.python-version == '3.12' && runner.os == 'Linux'
name: Lint
run: hatch run lint:all
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Run tests
run: hatch run cov
- name: Install Hatch
run: pip install --upgrade hatch

- if: matrix.python-version == '3.12' && runner.os == 'Linux'
name: Report Coveralls
uses: coverallsapp/github-action@v2
- name: Run tests
run: hatch run test
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,4 @@ cython_debug/
# IDEs and editors
.idea/
.vscode/
.zed/
12 changes: 12 additions & 0 deletions docs/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@

::: banks.prompt.AsyncPrompt

::: banks.registries.directory.DirectoryPromptRegistry
options:
inherited_members: true

::: banks.registries.file.FilePromptRegistry
options:
inherited_members: true

::: banks.registries.redis.RedisPromptRegistry
options:
inherited_members: true

## Default macros

Banks' package comes with default template macros you can use in your prompts.
Expand Down
62 changes: 58 additions & 4 deletions docs/registry.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,63 @@
## Prompt registry (BETA)

Prompt registry is a storage API for versioned prompts. It allows you to store and retrieve prompts from local storage.
Currently, it supports storing templates in a single JSON file or in a file system directory, but it can be extended to
support other storage backends.
The Prompt Registry provides a storage API for managing versioned prompts. It allows you to store and retrieve prompts from different storage backends. Currently, Banks supports two storage implementations:

- Directory-based storage
- Redis-based storage

### Usage

Coming soon.
```python
from banks import Prompt
from banks.registries.directory import DirectoryPromptRegistry
from pathlib import Path

# Create a registry
registry = DirectoryPromptRegistry(Path("./prompts"))

# Create and store a prompt
prompt = Prompt(
text="Write a blog post about {{topic}}",
name="blog_writer",
version="1.0",
metadata={"author": "John Doe"}
)
registry.set(prompt=prompt)

# Retrieve a prompt
retrieved_prompt = registry.get(name="blog_writer", version="1.0")
```

### Directory Registry

The DirectoryPromptRegistry stores prompts as individual files in a directory. Each prompt is saved as a `.jinja` file with the naming pattern `{name}.{version}.jinja`.

```python
# Initialize directory registry
registry = DirectoryPromptRegistry(
directory_path=Path("./prompts"),
force_reindex=False # Set to True to rebuild the index
)
```

### Redis Registry

The RedisPromptRegistry stores prompts in Redis using a key-value structure.

```python
from banks.registries.redis import RedisPromptRegistry

registry = RedisPromptRegistry(
redis_url="redis://localhost:6379",
prefix="banks:prompt:"
)
```

### Common Features

Both implementations support:

- Versioning with automatic "0" default version
- Overwrite protection with `overwrite=True` option
- Metadata storage
- Error handling for missing/invalid prompts
29 changes: 15 additions & 14 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ theme:
scheme: slate

nav:
- Home: 'index.md'
- Examples: 'examples.md'
- Python API: 'python.md'
- Prompt API: 'prompt.md'
- Configuration: 'config.md'
- Prompt Registry: 'registry.md'
- Home: "index.md"
- Examples: "examples.md"
- Python API: "python.md"
- Prompt API: "prompt.md"
- Configuration: "config.md"
- Prompt Registry: "registry.md"

plugins:
- search
Expand All @@ -22,13 +22,14 @@ plugins:
python:
paths: [src]
options:
docstring_style: google
show_root_heading: true
show_root_full_path: true
show_symbol_type_heading: true
show_source: false
show_signature_annotations: true
show_bases: false
docstring_style: google
show_root_heading: true
show_root_full_path: true
show_symbol_type_heading: false
show_source: false
show_signature_annotations: true
show_bases: false
separate_signature: true

markdown_extensions:
- attr_list
Expand All @@ -42,4 +43,4 @@ markdown_extensions:
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
- admonition
- admonition
Loading

0 comments on commit def2c31

Please sign in to comment.