support Jinja templating in docstring

[helmanofer]

Inspired from https://github.com/normal-computing/outlines#prompting
I am working with OpenAI and Bedrock (Claude)

Claude requires tweaking sometimes. for example:

Create a list of numbers from 0 to 5
{% if  clause  %}
Assistant: 
{% endif %}

[matankley]

@helmanofer I think the original initiative was to "hide" the model agnostic prompt templating within the task operators and let users just write business prompt logic.

Regarding the Claude example, would a claude task operator that implements the system/user/assistant prompt template that claude supports solve your need ? or you think there a need to handle prompt templates by yourself ?

[matankley]

@helmanofer But nevertheless, we should still think of how integrating the templates, because I guess there are plenty of use cases it might be beneficial

[matankley]

@helmanofer I start working on it now :) will update how that goes

[matankley]

Wow this is amazing! I loved it ! Enjoyed going over your code, and already noticed few things we can definitely integrate into declarai.

1. Jinja templating in general:
   I really liked the clean api of jinja=True

So I guess this looks good as well, I tested it and it works the same, preparing the MR right now.

@gpt_35.task(jinja=True)
def say_something(text: str) -> List[str]:
    """
    Extract all phone numbers from the given text: "{{ text }}"
    :return: list of numbers
    """


say_something.compile(text="Hey please reach me on +972-11111111")
say_something(text="Hey please reach me on +972-1111")

2. OpenAI functions as an alternative for getting the right structure (and put it as default when working with openai).

I generally think functions is the way to go and I'm really interested in integrating them. Will try to get it after we integrate the jinja.

3. Define more than one user prompt - being able to control seris of prompts in the docstring.

This is very powerful but I'm not sure yet what would be the best interface for that. Are you satisfied with the ">>" format in pydantic-prompter?

[helmanofer]

Thanks:)
No, I am not very satisfied with >> I had to find something that has small chance to be in YML.
Can you think of anything else?

[giladbarnea]

Hi, @helmanofer pydantic-prompter is really cool! Regarding the >> separator, what do you think about treating the whole docstring as a yaml file? Does that sound more convenient to you?

@Prompter(llm="openai", jinja=True, model_name="gpt-3.5-turbo-16k")
def rank_recommendation_entries(
        json_entries, user_query
) -> RecommendationResults:
    """
    - system: You are a movie ranking expert
    - user: >
        Which of the following JSON entries fit best to the query. 
        order by best fit descending
        Base your answer ONLY on the given JSON entries, 
        if you are not sure, or there are no entries
    - user: >
        The JSON entries:
        {{ json_entries }}
    - user: "query: {{ user_query }}"
    """

It would also allow yaml-parsing the string once, and not line-by-line. E.g

>>> yaml.safe_load(rank_recommendation_entries.__doc__)

[
    { 'system': 'You are a movie ranking expert' },
    { 'user': 'Which of the following JSON entries fit best ...\n' },
    { 'user': 'The JSON entries: {{ json_entries }}\n' },
    { 'user': 'query: {{ user_query }}' }
]

[helmanofer]

I tried :(
It causes many issues with white space alignment
In the example, you will have to write
{{ json_entries|indent(8) }}
Sometimes it gets worst

[helmanofer]

wait, this is a good idea, safe loading prior jinjaing, mmmm