safety_settings doesn't work for Gemini

[MichaelZhangCN]

Hi I tested Gemini find the safety setting does't work.

This is the config.yaml file:
litellm_settings:
drop_params: True
set_verbose: False
vertex_project: ""
vertex_location: "us-central1"

general_settings:
master_key: sk-1234

model_list:

• model_name: gemini-1.0-pro-001
  litellm_params:
  model: vertex_ai/gemini-1.0-pro-001
  temperature: 0.5
  safety_settings:
  • category: HARM_CATEGORY_HARASSMENT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_HATE_SPEECH
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_DANGEROUS_CONTENT
    threshold: BLOCK_ONLY_HIGH
• model_name: gemini-1.0-pro-002
  litellm_params:
  model: vertex_ai/gemini-1.0-pro-002
  temperature: 0.5
  safety_settings:
  • category: HARM_CATEGORY_HARASSMENT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_HATE_SPEECH
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_DANGEROUS_CONTENT
    threshold: BLOCK_ONLY_HIGH
• model_name: gemini-pro-1.5
  litellm_params:
  model: vertex_ai/gemini-1.5-pro-preview-0409
  temperature: 0.5
  safety_settings:
  • category: HARM_CATEGORY_HARASSMENT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_HATE_SPEECH
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    threshold: BLOCK_ONLY_HIGH
  • category: HARM_CATEGORY_DANGEROUS_CONTENT
    threshold: BLOCK_ONLY_HIGH

And this is the curl command:
curl --location '/completions'
--header 'Authorization: Bearer sk-1234'
--header 'Content-Type: application/json'
--data '{
"model": "gemini-1.0-pro-001",
"prompt": "You are a language translation master with ten years of translation experience. You need to translate the content provided by the user as close as possible to the language habits of the English-speaking country according to the user'''s requirements. You should retain the original punctuation marks. The user has given reference terms. If the content involves terminology When using a word, you must refer to the correct translation of the word term. For other words that are not within the scope, you only need to output the translation result without any additional explanation or explanation.\n Content:到我这边来，不然你们都会被强风刮走的！ Terminology:"
}'

This curl will return failure due to the safety filter of Gemini. But in Google Cloud Gemini console, when I set the safety filter to BLOCK FEW, it will return successfully.

I suspect the safety filter doesn't work when I use litellm

[MichaelZhangCN]

in proxy mode, seems satety filter is not used in Proxy async completions

[Manouchehri]

Converted to an issue here to track: #3318