[Bugfix]Start Failed with param --limit-mm-per-prompt image=N for Molmo #10028
Conversation
当启动molmo模型时候如果带上 --limit-mm-per-prompt image=N 参数 vllm启动时在dummy_data_for_profiling里的assert num_items >= num_expected 会失败 因为molmo模型定义的输入和一般多模态不一致详细可见molmo.py dummy_data_for_molmo方法 这个改进对于输入图片个数做出正确判断
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
|
According to the supported models page, we don't support multi-image input for this model yet. Are you willing to update the implementation of this model? |
i found that this model Molmo may have already support multi-image input. res = client.chat.completions.create(
model=MODEL_NAME,
messages=[
{
"role": "user",
"content": [{
"type": "image_url",
"image_url": {
"url": image_to_base64("case/case2.jpg")
}
},
{
"type": "image_url",
"image_url": {
"url": image_to_base64("case/case1.jpg")
}
},
{
"type": "text",
"text": "describe the pictures i show you"
}]
}
]
)
# it success return and give back description of 2 imagesi think this model has already support mliti-image .but i am not sure. related code is below . |
| @@ -244,6 +244,10 @@ def dummy_data_for_profiling( | |||
| if dummy_data.multi_modal_data is not None: | |||
| for k, v in dummy_data.multi_modal_data.items(): | |||
| num_items = len(v) if isinstance(v, list) else 1 | |||
| #fix for molmo | |||
| if k == "image" and model_config.hf_config.model_type == "molmo": | |||
There was a problem hiding this comment.
In any case, let's not hard code this here. Can you instead modify the dummy data inside the model's file?
Fix bug run api_server with param --limit-mm-per-prompt image=N for model_type molmo
in function dummy_data_for_profiling .there is a assertion num_items >= num_expected
which is always failed because in molmo model, milti-modal image is in a dict of only one item 'images'
you can find it in file molmo.py . function dummy_data_for_molmo
this commit make num_items correct and make vllm start correct.