Skip to content

Commit

Permalink
Update Help Text for --gpu-memory-utilization Argument (vllm-project#…
Browse files Browse the repository at this point in the history
  • Loading branch information
SuhongMoon authored Dec 18, 2023
1 parent 1b7c791 commit 290e015
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions vllm/engine/arg_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,13 @@ def add_cli_args(
type=int,
default=EngineArgs.swap_space,
help='CPU swap space size (GiB) per GPU')
parser.add_argument('--gpu-memory-utilization',
type=float,
default=EngineArgs.gpu_memory_utilization,
help='the percentage of GPU memory to be used for '
'the model executor')
parser.add_argument(
'--gpu-memory-utilization',
type=float,
default=EngineArgs.gpu_memory_utilization,
help='the fraction of GPU memory to be used for '
'the model executor, which can range from 0 to 1.'
'If unspecified, will use the default value of 0.9.')
parser.add_argument('--max-num-batched-tokens',
type=int,
default=EngineArgs.max_num_batched_tokens,
Expand Down

0 comments on commit 290e015

Please sign in to comment.