[Bug]: Facing deadlocks on LiteLLM

[xyannie]

What happened?

Version : v1.52.10

Ran some load tests which will hit litellm endpoint , observed occasionally will hit the following error

Relevant log output

ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError { code: "40P01", message: "deadlock detected", severity: "ERROR", detail: Some("Process 4199 waits for ShareLock on transaction 29865236; blocked by process 4225.
Process 4225 waits for ShareLock on transaction 29865235; blocked by process 4199."), column: None, hint: Some("See server log for query details.") }), transient: false })

Twitter / LinkedIn details

No response

[krrishdholakia]

ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError { code: "40P01", message: "deadlock detected", severity: "ERROR", detail: Some("Process 4199 waits for ShareLock on transaction 29865236; blocked by process 4225.

this deadlock seems to be from the prisma client writing to the db

can you please share steps to repro?

[xyannie]

I'll get back to you on litellm application side, but I conducted a load test on our GenAI models to assess the current throughput and thresholds. In our case, the workflow involves an agent invoking multiple tools (which may include additional LLM calls) to generate a final output.

Do let me know if there any additional information you need on our end.