[v0.1]Routing Logic:Load and latency-aware endpoint resilience #35
Description
Acceptance
- Endpoint selection using request concurrency and/or TTFT/TPOT. Using SLO driven metrics to automatic failover with load weighted selection between redundant endpoints
- circuit breaker with error rate and load signal deviation thresholds.