WIP litellm integration #320
Conversation
There was a problem hiding this comment.
I think it generally looks good:
- I wonder if there is a way to load the secrets directly from env vars instead of having to provide them in the function call.
- I'd also like to simplify passing in extra arguments but idk if sql functions allow
*argsor**kwargsI guess not
| ( %s | ||
| , 'hello world' | ||
| , api_key=>%s | ||
| , extra_options => jsonb_build_object('input_type', %s::text) |
There was a problem hiding this comment.
I guess there is no way of directly taking any additional arguments as an "extra option"? without wrapping it in a jsonb and passing it as an extra param?
| , extra_options => jsonb_build_object( | ||
| 'aws_access_key_id', %s::text, | ||
| 'aws_secret_access_key', %s::text, | ||
| 'aws_region_name', %s::text |
There was a problem hiding this comment.
Is there any way to autoload these from env vars if they exist? same for the api_key
| select vector_dims | ||
| ( | ||
| ai.litellm_embed | ||
| ( %s |
There was a problem hiding this comment.
I kinda wonder why litellm has no provider argument, I feel like model names will collide eventually.
| plpy.error(f"missing {secret_name} secret") | ||
| # This line should never be reached, but it's here to make the type checker happy. | ||
| return "" | ||
| return None |
There was a problem hiding this comment.
is there existing code that depends on this condition throwing an error?
There was a problem hiding this comment.
I don't know. I still need to do a full audit of all of the callers.
|
lookin' good |
TODOs: - tests for all integrations - fine-tune configuration parameters - fine-tune huggingface (cold-start?)
JamesGuthrie
force-pushed
the
jg/litellm-vectorizer
branch
from
ea8e0bf to
366ae89
Compare
No description provided.