Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llama-3.2-11B-Vision-Instruct来测评MMStar和HallusionBench #685

Closed
wangchunhui06 opened this issue Dec 23, 2024 · 2 comments
Closed

Llama-3.2-11B-Vision-Instruct来测评MMStar和HallusionBench #685

wangchunhui06 opened this issue Dec 23, 2024 · 2 comments
Assignees
@FangXinyu-0913

Comments

@wangchunhui06
Copy link

用Llama-3.2-11B-Vision-Instruct来测评MMStar和HallusionBench时，模型输出句子不完整（生成的prediction），大多数分词之后集中在130token。但是测MMVet时，就不会出现这种情况。
@FangXinyu-0913
Copy link
Collaborator

您好，针对mcq和y/n数据集，我们将max_new_token设定成了128，您可以在https://github.com/open-compass/VLMEvalKit/blob/main/vlmeval/vlm/llama_vision.py#L200进行修改

@wangchunhui06
Copy link
Author

知道啦，谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FangXinyu-0913 FangXinyu-0913

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@FangXinyu-0913 @wangchunhui06