running litellm behind proxy server

[amirclam]

I am running litellm behind proxy server using local text generation inference model

Setup:
model_list:
- model_name: star-coder
  litellm_params:
    model: huggingface/my-model
    api_base: [{local model adrress}]
    api_key: no-key
    caching: False
litellm_settings:
  json_logs: true
  cache: True
  telemetry: False
general_settings:
  alerting_threshold: 1
  alerting: ["email"]
  otel: False
  disable_spend_logs: True
  proxy_batch_write_at: 120  

when running it with our setting https_proxy every call get stuck for 5 minutes

even due my model and redis are inside my k8s cluster

what configuration am i missing ?

[ishaan-jaff]

Hi @amirclam - can we hop on a call to debug this. Hoping to get steps to repro the issue

my cal for your convenience: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
my linkedin if you prefer DMs: https://www.linkedin.com/in/reffajnaahsi/