feat: GPT35Generator

[ZanSara]

Related Issues

• related to OpenAI LLM Generators for Haystack 2.0 #5623

Proposed Changes:

• Add ChatGPTGenerator
• Add unit tests
• Add end to end tests

How did you test it?

• Local tests run
• CI

Notes for the reviewer

openai already added to haystack-ai: deepset-ai/haystack-preview-package#2

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[mathislucka]

A small note on naming, a ChatGPTGenerator that does not support chat feels a bit confusing to me.

[ZanSara]

Good point 👀 Things should clear up as soon as the Agents proposal takes shape. At that point we will rework this very simple component into something more powerful and/or clarify better what it can and can't do. It is in no way feature complete, final or anything. It's just a simple component to make RAG possible in v2. Let's keep iterating!

---

@mathislucka @sjrl I might rename it GPT35Generator if that helps, but it looks a bit uglier to me. Wdyt?

[mathislucka]

@mathislucka @sjrl I might rename it GPT35Generator if that helps, but it looks a bit uglier to me. Wdyt?

That sounds better I think.

[dfokina]

@ZanSara hey I see there are some more places with ChatGPT name, like the name of the .py file or in docstrings 🙂

[ZanSara]

Thank you! I'm having another pass 🙌

[anakin87]

Great work!

I took a general look and I have some high-level doubts.
(Probably many of these points are about legitimate implementation choices or future developments to be done in separate PRs...)

1. Although it has already been discussed, the naming sounds strange to me. I would not expect that I can use the GPT35Generator to run GPT4. Other options: OpenAIGenerator, OpenAIGPTGenerator,...

2. (Partly related to the previous point?) This component does not support the legacy Completions API, supported in Haystack 1.x. Although declared legacy, the Completions API will be probably retired in 2024. Don't we want to support it in Haystack 2.x? (It may well be a reasonable choice)

3. What happens if the user enters a wrong model name? Is it ok to let the OpenAI SDK raise an error or do we want to intercept it in Haystack?

4. What happens if the user sends a prompt that fills/exceeds the context window?

5. Maybe the OpenAI organization should be declared in __init__ and then used. See feat: support OpenAI-Organization for authentication #5292.

6. Azure ChatGPT Generator (corresponding to this invocation layer) will be addressed in another PR. Right?

[ZanSara]

Ok let's take them one by one:

1. Although it has already been discussed, the naming sounds strange to me. I would not expect that I can use the GPT35Generator to run GPT4. Other options: OpenAIGenerator, OpenAIGPTGenerator,...

I plan to make a subclass called GPT4Generator that simply sets another default model. I thought about OpenAIGenerator, but there are models that have a very different interface (think about davinci, if we ever want to support it) and in any case if tomorrow OpenAI releases another model we'll have naming issues. In any case rest assured that at least the GPT4 alias is coming asap.

2. (Partly related to the previous point?) This component does not support the legacy Completions API, supported in Haystack 1.x. Although declared legacy, the Completions API will be probably retired in 2024. Don't we want to support it in Haystack 2.x? (It may well be a reasonable choice)

If we do, I'd rather support them into a dedicated generator that we will be able to deprecate easily when the time comes.

3. What happens if the user enters a wrong model name? Is it ok to let the OpenAI SDK raise an error or do we want to intercept it in Haystack?

Good point! I'll check how the error looks like. I expect it to be quite clear, if it is I'll let it go across. Otherwise I'll figure out a way to communicate it to the user better..

4. What happens if the user sends a prompt that fills/exceeds the context window?

Right! I'll add a test!

5. Maybe the OpenAI organization should be declared in __init__ and then used. See feat: support OpenAI-Organization for authentication #5292.

Ah right! It got lost in the last refactoring. Let me bring it back.

6. Azure ChatGPT Generator (corresponding to this invocation layer) will be addressed in another PR. Right?

Yes Azure support is for the next PR 🙂

[anakin87]

👍