Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Usage]: Can it support using the siglip model as the vision model in a multimodal model? #7144

Closed
BrenchCC opened this issue Aug 5, 2024 · 3 comments · Fixed by #6942
Closed
Labels
usage How to use vllm

Comments

@BrenchCC
Copy link

BrenchCC commented Aug 5, 2024

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.

@BrenchCC BrenchCC added the usage How to use vllm label Aug 5, 2024
@jeejeelee
Copy link
Collaborator

jeejeelee commented Aug 5, 2024

FYI: #6942

@ywang96
Copy link
Member

ywang96 commented Aug 5, 2024

Just as a reference, generally speaking, you're welcome to import any ViT directly from transformers if the VLM utilizes it (but it would be also great if you port the ViT over)

@BrenchCC
Copy link
Author

BrenchCC commented Aug 5, 2024

Thank you so much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
usage How to use vllm
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants