function_calling #1100
Unanswered
crystal-ltf
asked this question in
Q&A
function_calling
#1100
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
使用vllm 0.6.2，运行qwen2.5-72b-instruct-AWQ，脚本为vllm serve Qwen2.5-72B-Instruct-AWQ --enable-auto-tool-choice --tool-call-parser hermes --host 0.0.0.0 --port 25020 --tensor-parallel-size 4 --max-model-len 10240 --gpu-memory-utilization 0.8，无法进行工具调用，后来又试了qwen2.5-72B-instruct是可以的，所以AWQ版本是不支持工具调用吗？
Beta Was this translation helpful? Give feedback.
All reactions