Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix]Start Failed with param --limit-mm-per-prompt image=N for Molmo #10028

Closed
wants to merge 2 commits into from
Closed

[Bugfix]Start Failed with param --limit-mm-per-prompt image=N for Molmo #10028

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b54e6c4
[Bugfix]Start Failed with param --limit-mm-per-prompt image=N for Molmo
Nov 5, 2024
88e8c8b
Merge branch 'vllm-project:main' into main
congwupiece Nov 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions vllm/inputs/registry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,10 @@
if dummy_data.multi_modal_data is not None:
for k, v in dummy_data.multi_modal_data.items():
num_items = len(v) if isinstance(v, list) else 1
#fix for molmo
if k == "image" and model_config.hf_config.model_type == "molmo":

Check failure on line 248 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.8)

Ruff (E501) 

vllm/inputs/registry.py:248:81: E501 Line too long (81 > 80)

Check failure on line 248 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.9)

Ruff (E501) 

vllm/inputs/registry.py:248:81: E501 Line too long (81 > 80)

Check failure on line 248 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.10)

Ruff (E501) 

vllm/inputs/registry.py:248:81: E501 Line too long (81 > 80)

Check failure on line 248 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.11)

Ruff (E501) 

vllm/inputs/registry.py:248:81: E501 Line too long (81 > 80)

Check failure on line 248 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.12)

Ruff (E501) 

vllm/inputs/registry.py:248:81: E501 Line too long (81 > 80)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, let's not hard code this here. Can you instead modify the dummy data inside the model's file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

if isinstance(v, dict) and "images" in v:

Check failure on line 249 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.8)

Ruff (SIM102) 

vllm/inputs/registry.py:248:17: SIM102 Use a single `if` statement instead of nested `if` statements

Check failure on line 249 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.9)

Ruff (SIM102) 

vllm/inputs/registry.py:248:17: SIM102 Use a single `if` statement instead of nested `if` statements

Check failure on line 249 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.10)

Ruff (SIM102) 

vllm/inputs/registry.py:248:17: SIM102 Use a single `if` statement instead of nested `if` statements

Check failure on line 249 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.11)

Ruff (SIM102) 

vllm/inputs/registry.py:248:17: SIM102 Use a single `if` statement instead of nested `if` statements

Check failure on line 249 in vllm/inputs/registry.py

View workflow job for this annotation

GitHub Actions / ruff (3.12)

Ruff (SIM102) 

vllm/inputs/registry.py:248:17: SIM102 Use a single `if` statement instead of nested `if` statements
num_items = len(v["images"])
num_expected = mm_counts[k]
assert num_items >= num_expected, (
f"Expected at least {num_expected} dummy '{k}' instances "
Expand Down
Loading