Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

如何用benchmark来评测Qwen-7B #439

Closed
2 tasks done
cgq0816 opened this issue Sep 19, 2023 · 2 comments
Closed
2 tasks done

如何用benchmark来评测Qwen-7B #439

cgq0816 opened this issue Sep 19, 2023 · 2 comments
Assignees

Comments

@cgq0816
Copy link

cgq0816 commented Sep 19, 2023

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.

Describe the bug

评测方法中需要输入/path/to/tokenizer/tokenizer.model ,但是Qwen-7B不会生成tokenizer.model ,请问我该如何进行评测呢?

Reproduction

python profile_restful_api.py http://0.0.0.0:33337 /data/cgq/workspace/triton_models/tokenizer/tokenizer.model /home/gqchen/ShareGPT_V3_unfiltered_cleaned_split.json --concurrency 5

Error traceback

No response

@lvhan028
Copy link
Collaborator

忘记qwen是tikoken了。
感觉我们得改下restful_api的profile方式。我们内部先讨论下,有结论后再来同步

@AllentDan
Copy link
Collaborator

Hi, @cgq0816 the bug was fixed in #443

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants