[GuideLLM Refactor] scheduler package updates, rewrites, and tests expansion

[markurtz]

Summary

Introduces a comprehensive constraints system and enhanced timing control for the scheduler refactor. The implementation moves from hardcoded execution limits to a flexible, composable constraint system that enables sophisticated benchmark stopping criteria. Additionally, request timing calculations are moved from precalculated to per-request basis, enabling dynamic rate adjustments and better distributed coordination.

Details

• Added constraints system (constraints.py): Implements Protocol-based constraint architecture with support for request limits, duration limits, error thresholds, and sliding window error rates
  • MaxNumberConstraint: Limits execution based on request count
  • MaxDurationConstraint: Limits execution based on time duration
  • MaxErrorsConstraint: Limits execution based on absolute error count
  • MaxErrorRateConstraint: Limits execution based on sliding window error rate
  • MaxGlobalErrorRateConstraint: Limits execution based on global error rate
  • ConstraintsInitializerFactory: Registry system for constraint creation and serialization
• Refactored core objects (objects.py): Replaced result.py and expanded capabilities
  • Made scheduler package fully generic, decoupling from backend-specific types
  • Added BackendInterface protocol for type-safe backend integration
  • Enhanced ScheduledRequestInfo with comprehensive timing and status tracking
  • Added SchedulerState for distributed state coordination
  • Introduced SchedulerUpdateAction for constraint-based control signals
• Enhanced scheduling strategies (strategy.py): Introduced request timing abstractions
  • Added ScheduledRequestTimings base class for timing implementations
  • LastCompletionRequestTimings: For synchronous and concurrent strategies
  • NoDelayRequestTimings: For maximum throughput strategies
  • ConstantRateRequestTimings: For fixed-rate scheduling
  • PoissonRateRequestTimings: For stochastic request patterns
  • Strategies now create per-worker timing instances instead of precalculated schedules
• Added environment abstractions (environment.py): Coordination layer for distributed execution
  • Environment protocol for distributed synchronization
  • NonDistributedEnvironment implementation for single-node execution
• Worker process management (worker.py, worker_group.py): Distributed request processing infrastructure
  • Individual worker process management with lifecycle coordination
  • Multi-process orchestration with state synchronization
  • Constraint evaluation and graceful shutdown coordination

Test Plan

• Full unit tests and some integration tests added and passing

Related Issues

• Part of scheduler refactor initiative to support distributed benchmarking

---

• "I certify that all code in this PR is my own, except as noted below."

Use of AI

• Includes AI-assisted code completion
• Includes code generated by an AI application
• Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

[Copilot]

Pull Request Overview

This PR introduces a comprehensive refactor of the scheduler package to support distributed benchmarking with a flexible constraint system and enhanced timing control. The implementation moves from hardcoded execution limits to a composable constraint architecture and changes request timing calculations from precalculated to per-request basis for dynamic rate adjustments.

Key changes:

• Added a flexible constraints system with Protocol-based architecture supporting request limits, duration limits, and error thresholds
• Refactored core scheduler objects to be fully generic and decoupled from backend-specific types
• Enhanced scheduling strategies with per-worker timing instances instead of precalculated schedules

Reviewed Changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/scheduler/test_worker_group.py	Comprehensive test coverage for WorkerProcessGroup multi-process orchestration
tests/unit/scheduler/test_worker.py	Full test suite for WorkerProcess individual worker functionality
tests/unit/scheduler/test_strategy.py	Extensive testing of scheduling strategies and timing implementations
tests/unit/scheduler/test_scheduler.py	Core scheduler singleton testing with distributed coordination
tests/unit/scheduler/test_objects.py	Complete test coverage for scheduler objects and type relationships
tests/unit/scheduler/test_environment.py	Testing of environment abstractions for distributed coordination
tests/unit/scheduler/test_constraints.py	Comprehensive constraint system testing with factory patterns
src/guidellm/scheduler/worker_group.py	Multi-process worker orchestration implementation
src/guidellm/scheduler/worker.py	Individual worker process management implementation
src/guidellm/scheduler/strategy.py	Request scheduling strategies with timing abstractions
src/guidellm/scheduler/scheduler.py	Thread-safe singleton scheduler for distributed coordination
src/guidellm/scheduler/result.py	File removed as functionality moved to objects.py
src/guidellm/scheduler/queues.py	File removed as functionality integrated into worker components

Comments suppressed due to low confidence (2)

tests/unit/scheduler/test_objects.py:1

• This test expects TypeError but ScheduledRequestInfo has Field defaults defined for all required fields. The test should either provide invalid field types or check for ValidationError instead.

from __future__ import annotations

tests/unit/scheduler/test_objects.py:1

• The field name created_at doesn't exist in ScheduledRequestInfo. This should be scheduler_start_time based on the model definition.

from __future__ import annotations

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}