[Bug]: H Using athina as the success_callback will cause CPU usage to reach 90%

[Caixiaopig]

What happened?

In config.yaml, I set litellm_settings: success_callback to athina and found that the CPU usage was abnormal, even reaching up to 90%. Meanwhile, I also tested other success_callback providers like langfuse and promptlayer, and they did not encounter this issue.

code example

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
litellm_settings:
  success_callback: ["athina"]

Relevant log output

No response

Twitter / LinkedIn details

No response

[ishaan-jaff]

This looks like an issue related to athina specifically

[ishaan-jaff]

making discussions since not a litellm issue

[ishaan-jaff]

@vivek-athina can you take a look at this

[vivek-athina]

I used the same config and ran multiple requests. The CPU usage never reached 90%, infact it was not abnormal at all.
Config used

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
litellm_settings:
  success_callback: ["athina"]

curl command used

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "user",
            "content": "which llm are you"
        }
    ]
}'

Video on my test run where the usage never went above 90%

Screenshare.-.2024-04-17.5_58_42.PM.mp4

[ishaan-jaff]

@vivek-athina I think you need to run 20-30 RPS - your test looked like you were doing 1 RPS

[ishaan-jaff]

@Caixiaopig can you share what load you saw this on ?