Skip to content

Commit

Permalink
Update kv_cache_manager.py
Browse files Browse the repository at this point in the history
Co-authored-by: Woosuk Kwon <[email protected]>
  • Loading branch information
comaniac and WoosukKwon authored Dec 31, 2024
1 parent 44e63ea commit 5c5e52e
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions vllm/v1/core/kv_cache_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,8 @@ def allocate_slots(
# If a computed block of a request is an eviction candidate (in the
# free queue and ref_cnt == 0), it cannot be counted as a free block
# when allocating this request.
num_evictable_computed_blocks = len(
[blk for blk in computed_blocks if blk.ref_cnt == 0])
num_evictable_computed_blocks = sum(
blk for blk in computed_blocks if blk.ref_cnt == 0)

Check failure on line 207 in vllm/v1/core/kv_cache_manager.py

View workflow job for this annotation

GitHub Actions / mypy (3.9)

Generator has incompatible item type "KVCacheBlock"; expected "bool" [misc]

Check failure on line 207 in vllm/v1/core/kv_cache_manager.py

View workflow job for this annotation

GitHub Actions / mypy (3.10)

Generator has incompatible item type "KVCacheBlock"; expected "bool" [misc]

Check failure on line 207 in vllm/v1/core/kv_cache_manager.py

View workflow job for this annotation

GitHub Actions / mypy (3.11)

Generator has incompatible item type "KVCacheBlock"; expected "bool" [misc]

Check failure on line 207 in vllm/v1/core/kv_cache_manager.py

View workflow job for this annotation

GitHub Actions / mypy (3.12)

Generator has incompatible item type "KVCacheBlock"; expected "bool" [misc]

num_required_blocks = cdiv(num_tokens, self.block_size)
if (num_required_blocks > self.free_block_queue.num_free_blocks -
Expand Down

0 comments on commit 5c5e52e

Please sign in to comment.