Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
[Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API S…
Browse files Browse the repository at this point in the history
  • Loading branch information
ywang96 authored and robertgshaw2-redhat committed Jun 11, 2024
1 parent 8f865f6 commit d3bd135
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions vllm/entrypoints/openai/api_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,16 @@ async def authentication(request: Request, call_next):
served_model_names = [args.model]

engine_args = AsyncEngineArgs.from_cli_args(args)

# Enforce pixel values as image input type for vision language models
# when serving with API server
if engine_args.image_input_type is not None and \
engine_args.image_input_type.upper() != "PIXEL_VALUES":
raise ValueError(
f"Invalid image_input_type: {engine_args.image_input_type}. "
"Only --image-input-type 'pixel_values' is supported for serving "
"vision language models with the vLLM API server.")

engine = AsyncLLMEngine.from_engine_args(
engine_args, usage_context=UsageContext.OPENAI_API_SERVER)

Expand Down

0 comments on commit d3bd135

Please sign in to comment.