glm-4v模型只传文字报KeyError: 'images'，同时传文字和图片不报错

[JumpNew]

System Info / 系統信息

glm-4v/transformer/9b/8bit

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

v1.0.0

The command used to start Xinference / 用以启动 xinference 的命令

只传文本，不传图片的时候，报错，同时传文本和图片是不报错的

2024-11-17 02:42:30,493 xinference.model.llm.transformers.core 156 ERROR prepare inference error with
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 283, in getattr
return self.data[item]
KeyError: 'images'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/transformers/core.py", line 696, in prepare_batch_inference
r.full_prompt = self._get_full_prompt(r.prompt, tools)
File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/transformers/glm4v.py", line 228, in _get_full_prompt
"images": inputs.images.squeeze(0),
File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 285, in getattr
raise AttributeError
AttributeError
ERROR:xoscar.api:Destroy generator a547d860a4d011efafd80242ac110005 due to an error encountered.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 419, in xoscar_next
r = await asyncio.create_task(_async_wrapper(gen))
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 409, in _async_wrapper
return await _gen.anext() # noqa: F821
File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 475, in _to_async_gen
async for v in gen:
File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 671, in _queue_consumer
raise RuntimeError(res[len(XINFERENCE_STREAMING_ERROR_FLAG) :])
RuntimeError
2024-11-17 02:42:30,551 xinference.api.restful_api 1 ERROR Chat completion stream got an error: [address=0.0.0.0:39017, pid=156]
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/xinference/api/restful_api.py", line 2010, in stream_results
async for item in iterator:
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 340, in anext
return await self._actor_ref.xoscar_next(self._uid)
File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/context.py", line 231, in send
return self._process_result_message(result)
File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/context.py", line 102, in _process_result_message
raise message.as_instanceof_cause()
File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/pool.py", line 659, in send
result = await self._run_coro(message.message_id, coro)
File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/pool.py", line 370, in _run_coro
return await coro
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 384, in on_receive
return await super().on_receive(message) # type: ignore
File "xoscar/core.pyx", line 558, in on_receive
raise ex
File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive
async with self._lock:
File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
result = await result
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 431, in xoscar_next
raise e
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 419, in xoscar_next
r = await asyncio.create_task(_async_wrapper(gen))
File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 409, in _async_wrapper
return await _gen.anext() # noqa: F821
File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 475, in _to_async_gen
async for v in gen:
File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 671, in _queue_consumer
raise RuntimeError(res[len(XINFERENCE_STREAMING_ERROR_FLAG) :])
RuntimeError: [address=0.0.0.0:39017, pid=156]

Reproduction / 复现过程

只传文本的时候，报错。

Expected behavior / 期待表现

传文本的时候，也能正常响应。

[qinxuye]

应该和模型本身有关系，需要确认下 glm-4v 是不是支持不传图片。

[JumpNew]

这个应该支持传文本的