[Feature]: support tracking remaining tpm/rpm for gemini models

[krrishdholakia]

The Feature

Support tracking remaining tpm/rpm for gemini models

• allows prometheus metrics to capture this information

User config.yaml

model_list: 
   - model_name: gemini/*
      litellm_params: 
         model: gemini/*

Motivation, pitch

allows user to know how often they're hitting their rate limits on gemini

Twitter / LinkedIn details

No response

[krrishdholakia]

• add the router cache to the litellm logging object
• in litellm logging -> update when a successful user request is made with the tpm/rpm key containing the model id + model name
• on router success -> get_model_info on router should handle wildcard routes - add any litellm model cost map info to the user set config information -> return a model info object containing the gemini model-specific tpm/rpm limits
• get_tokens_and_requests(model_id: str, model: str, router: Router) -> returns the current tpm/rpm values for a given model_id + model name
• get_remaining_tokens_and_requests(model_id: str, model: str, router: Router) -> returns the diff b/w the set limit and the current value

[krrishdholakia]

Todo:

• get_model_info on router should handle wildcard routes
• add tpm/rpm limits for all gemini models on model cost map
• update router deployment_callback_on_success and deployment_callback_on_failure to track tpm/rpm on success
• implement get_tokens_and_requests router.get_model_group_usage() already exists
• implement get_remaining_tokens_and_requests