[Feature]: Support n (number of completions) in together_ai models

[slobodan-nf]

The Feature

I am able to set n (number of completions) to a number greater than 1 when using the together API directly, but not through litellm. I am using Meta-Llama-3.1-8B-Instruct-Turbo.

Example using the together API:

from together import Together

client = Together(api_key="")

completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "What are the top 3 things to do in New York?"}],
    temperature=0.5,
    n=2,
)

print(len(completion.choices)) # prints 2

Example using the litellm API:

import litellm
import os

os.environ["TOGETHERAI_API_KEY"] = ""

completion = litellm.completion(
    model="together_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "What are the top 3 things to do in New York?"}],
    temperature=0.5,
    n=2,
)
print(len(completion.choices)) # raises an exception

The exception:

litellm.UnsupportedParamsError: together_ai does not support parameters: {'n': 2}, for model=meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo. To drop these, set `litellm.drop_params=True` or for proxy:

`litellm_settings:
 drop_params: true`

Motivation, pitch

I would like to use this feature as it saves costs, reduces latency and makes the implementation cleaner.

Twitter / LinkedIn details

www.linkedin.com/in/slobodan-jenko-466370172