Skip to content

Conversation

mxrcooo
Copy link

@mxrcooo mxrcooo commented Oct 5, 2025

Title

Ensure virtual-key model budgets force router fallbacks

Relevant issues

Fixes #15223

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix
✅ Test

Changes

  • Refactored async_log_success_event() to parse the real user_api_key_model_max_budget, resolve aliases, and record spend against the same cache keys that enforcement uses.
  • Implemented async_filter_deployments() to mirror the deployment-level limiter flow:
    1. Read the virtual key’s budget from request metadata.
    2. Resolve aliases (e.g., model_name → litellm_params.model) and check cached spend for each deployment.
    3. Drop deployments whose spend exceeds max_budget.
    4. Surface a BudgetExceededError only after every deployment has been filtered, matching existing auth behaviour.
  • Added unit tests to confirm spend tracking still works and that routing now excludes over-budget deployments (and raises when all fallbacks are exhausted).

Screenshot of passing tests + all make test-unit passes:
Screenshot 2025-10-06 at 01 19 32

Copy link

vercel bot commented Oct 5, 2025

@mxrcooo is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link

CLAassistant commented Oct 5, 2025

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Virtual Key Budget tracking does not work properly for routing
2 participants