Skip to content

Commit

Permalink
[BUG] fix the case when num_used_blocks < 0 (#1277)
Browse files Browse the repository at this point in the history
* when num_required_blocks > num_drop_blocks, num_used_blocks < 0

[Bug Fix]  when num_required_blocks > num_drop_blocks, num_used_blocks < 0. This behavior is wrong

* Update window_block_manager.py

fix num_used_blocks
  • Loading branch information
jjjjohnson authored Mar 13, 2024
1 parent a3fad24 commit 8d8be52
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions lmdeploy/pytorch/paging/block_manager/window_block_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,13 +118,16 @@ def __reuse_droped_blocks(num_required_blocks, num_drop_blocks,
"""reuse dropped blocks."""
num_used_blocks = min(num_drop_blocks - num_required_blocks,
num_required_blocks)
reused_blocks = droped_blocks[:num_used_blocks]
if num_used_blocks > 0:
reused_blocks = droped_blocks[:num_used_blocks]
else:
reused_blocks = droped_blocks
logical_blocks.append(reused_blocks)

num_drop_blocks = num_drop_blocks - num_used_blocks
if num_used_blocks > 0:
droped_blocks = droped_blocks[num_used_blocks:]
else:
num_used_blocks = num_drop_blocks
droped_blocks = None
num_required_blocks = num_required_blocks - num_used_blocks
return num_required_blocks, droped_blocks
Expand Down

0 comments on commit 8d8be52

Please sign in to comment.