Make funtion templates return templates to be rendered #22
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rlouf
force-pushed
the
template-as-functions
branch
from
1b914c1
to
fb5e355
Compare
Prompts remplates are currently contained in the docstring of decorated
functions. The main issue with this is that prompt templates cannot be
composed.
In this commit we instead require users to return the prompt
template from the function. The template will then automatically be
rendered using the values passed to the function. This is very flexible:
some variables can be used inside the functions and not be present in
the Jinja2 template that is returned, for instance:
```python
import prompts
@prompts.template
def my_template(a, b):
prompt = f'This is a first variable {a}'
return prompt + "and a second {{b}}"
```
rlouf
force-pushed
the
template-as-functions
branch
2 times, most recently
from
981f8cd
to
8ccc783
Compare
A concern is that rendering the template this way could be much slower, assuming Jinja2 makes extensive use of caching. We thus write a comparative benchmark between using Jinja2 templates or simply concatenating strings.
rlouf
force-pushed
the
template-as-functions
branch
from
8ccc783
to
b54601b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prompts templates are currently contained in the docstring of decorated functions. The main issue with this is that prompt templates cannot be composed.
In this commit we instead require users to return the prompt template from the function. The template will then automatically be rendered using the values passed to the function. This is very flexible: some variables can be used inside the functions and not be present in the Jinja2 template that is returned, for instance:
Closes #17.
Potential issue with performance
I have one reservation related to this change. dottxt-ai/outlines#1162 showed that we can dramatically increase rendering speed by compiling the template only once. Here we have no guarantee that the template being rendered will be the same between different calls.
We run a comparison benchmark between using
@prompt.templateand a simple function. The performance hit is real (using the prompt template is 1,000 times slower), but we are sub-microsecond so I am not sure this really matters. We added a note in the documentation.