Skip to content

Commit

Permalink
Adjust the comments
Browse files Browse the repository at this point in the history
  • Loading branch information
KuntaiDu committed Oct 25, 2024
1 parent 3607926 commit a9e1897
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions vllm/core/block/cpu_offloading_block_allocator.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,13 @@ def __init__(self, cpu_block_allocator: PrefixCachingBlockAllocator,
"""
GPU block should only be in one of the following three status:
uncached: allocated blocks that didn't hit any cache
cached: allocated blocks that hit the cache, either in GPU or in CPU
cached: allocated blocks that are cached, either in GPU or in CPU
free: the blocks are not allocated by block allocator
block allocator will automatically track free blocks, and we don't need
to specially handle cached blocks. So we only track uncached blocks
This implementation aims to transform uncacherd blocks to cached blocks
by performing GPU to CPU copy when calling `get_and_reset_swaps`
As block allocator will automatically track free blocks, and we don't
need to specially handle cached blocks. So we only track uncached blocks
"""
self._uncached_blocks: Deque[Block] = deque()
"""
Expand Down

0 comments on commit a9e1897

Please sign in to comment.