xinference无法正常运行

[ybsbbw]

System Info / 系統信息

linux环境下，8卡A800服务器

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

accelerate 1.1.1
aiobotocore 2.7.0
aiofiles 23.2.1
aiohappyeyeballs 2.4.3
aiohttp 3.11.7
aioitertools 0.12.0
aioprometheus 23.12.0
aiosignal 1.3.1
alabaster 0.7.12
altair 5.5.0
anaconda-client 1.9.0
anaconda-navigator 2.1.1
anaconda-project 0.10.1
annotated-types 0.7.0
anyio 4.6.2.post1
appdirs 1.4.4
argh 0.26.2
argon2-cffi 20.1.0
arrow 0.13.1
asn1crypto 1.4.0
astroid 2.6.6
astropy 4.3.1
async-generator 1.10
async-timeout 5.0.1
atomicwrites 1.4.0
attrs 24.2.0
autopep8 1.5.7
Babel 2.9.1
backcall 0.2.0
backports.functools-lru-cache 1.6.4
backports.shutil-get-terminal-size 1.0.0
backports.tempfile 1.0
backports.weakref 1.0.post1
bcrypt 4.2.1
beautifulsoup4 4.10.0
binaryornot 0.4.4
bitarray 2.3.0
BitVector 3.5.0
bkcharts 0.2
black 19.10b0
bleach 4.0.0
bokeh 2.4.1
boto 2.49.0
botocore 1.31.64
Bottleneck 1.3.2
brotlipy 0.7.0
cached-property 1.5.2
certifi 2021.10.8
cffi 1.14.6
chardet 4.0.0
charset-normalizer 2.0.4
civrealm 0.1.2
click 8.1.7
cloudpickle 2.0.0
clyent 1.2.2
colorama 0.4.4
conda 4.10.3
conda-build 3.21.5
conda-content-trust 0+unknown
conda-pack 0.6.0
conda-package-handling 1.7.3
conda-repo-cli 1.0.4
conda-token 0.3.0
conda-verify 3.4.2
contextlib2 0.6.0.post1
cookiecutter 1.7.2
cryptography 3.4.8
cycler 0.10.0
Cython 0.29.24
cytoolz 0.11.0
daal4py 2021.3.0
dask 2021.10.0
debugpy 1.4.1
decorator 5.1.0
defusedxml 0.7.1
diff-match-patch 20200713
distributed 2021.10.0
distro 1.9.0
docker 7.1.0
docutils 0.17.1
ecdsa 0.19.0
entrypoints 0.3
et-xmlfile 1.1.0
exceptiongroup 1.2.2
Farama-Notifications 0.0.4
fastapi 0.110.3
fastcache 1.1.0
ffmpy 0.4.0
filelock 3.16.1
flake8 3.9.2
Flask 1.1.2
fonttools 4.25.0
frozenlist 1.5.0
fsspec 2023.10.0
func-timeout 4.3.5
future 0.18.2
gevent 21.8.0
glob2 0.7
gmpy2 2.0.8
gradio 4.26.0
gradio_client 0.15.1
greenlet 1.1.1
grpcio 1.68.0
gymnasium 0.29.1
h11 0.14.0
h5py 3.3.0
HeapDict 1.0.1
html5lib 1.1
httpcore 1.0.7
httpx 0.27.2
huggingface-hub 0.26.2
idna 3.2
imagecodecs 2021.8.26
imageio 2.9.0
imagesize 1.2.0
importlib-metadata 4.8.1
importlib_resources 6.4.5
inflection 0.5.1
iniconfig 1.1.1
intervaltree 3.1.0
ipykernel 6.4.1
ipython 7.29.0
ipython-genutils 0.2.0
ipywidgets 7.6.5
isort 5.9.3
itsdangerous 2.0.1
jdcal 1.4.1
jedi 0.18.0
jeepney 0.7.1
Jinja2 2.11.3
jinja2-time 0.2.0
jiter 0.8.0
jmespath 1.0.1
joblib 1.1.0
json5 0.9.6
jsonschema 3.2.0
jupyter 1.0.0
jupyter-client 6.1.12
jupyter-console 6.4.0
jupyter-core 4.8.1
jupyter-server 1.4.1
jupyterlab 3.2.1
jupyterlab-pygments 0.1.2
jupyterlab-server 2.8.2
jupyterlab-widgets 1.0.0
keyring 23.1.0
kiwisolver 1.3.1
lazy-object-proxy 1.6.0
libarchive-c 2.9
llvmlite 0.37.0
locket 0.2.1
lxml 4.6.3
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.4.3
matplotlib-inline 0.1.2
mccabe 0.6.1
mdurl 0.1.2
mistune 0.8.4
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
mock 4.0.3
modelscope 1.20.1
more-itertools 8.10.0
mpmath 1.2.1
msgpack 1.0.2
multidict 6.1.0
multipledispatch 0.6.0
munkres 1.1.4
mypy-extensions 0.4.3
narwhals 1.14.2
navigator-updater 0.2.1
nbclassic 0.2.6
nbclient 0.5.3
nbconvert 6.1.0
nbformat 5.1.3
nest-asyncio 1.5.1
networkx 2.6.3
nltk 3.6.5
nose 1.3.7
notebook 6.4.5
numba 0.54.1
numexpr 2.7.3
numpy 1.22.4
numpydoc 1.1.0
nvidia-ml-py 12.560.30
olefile 0.46
openai 1.55.1
opencv-contrib-python 4.10.0.84
openpyxl 3.0.9
orjson 3.10.12
outcome 1.3.0.post0
overrides 7.7.0
packaging 21.0
pandas 1.3.4
pandocfilters 1.4.3
parso 0.8.2
partd 1.2.0
passlib 1.7.4
path 16.0.0
pathlib2 2.3.6
pathspec 0.7.0
patsy 0.5.2
peft 0.13.2
pep8 1.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.4.0
pip 21.2.4
pkginfo 1.7.1
pluggy 0.13.1
plumbum 1.8.2
ply 3.11
poyo 0.5.0
prometheus-client 0.11.0
prompt-toolkit 3.0.20
propcache 0.2.0
protobuf 5.28.3
psutil 6.1.0
ptyprocess 0.7.0
py 1.10.0
pyasn1 0.6.1
pycodestyle 2.7.0
pycosat 0.6.3
pycparser 2.20
pycurl 7.44.1
pydantic 2.10.2
pydantic_core 2.27.1
pydantic-extra-types 2.1.0
pydantic-settings 2.0.3
pydocstyle 6.1.1
pydub 0.25.1
pyerfa 2.0.0
pyflakes 2.3.1
Pygments 2.18.0
PyJWT 2.1.0
pylint 2.9.6
pyls-spyder 0.4.0
pynvml 11.5.3
pyodbc 4.0.0-unsupported
pyOpenSSL 21.0.0
pyparsing 3.0.4
PyQt5 5.15.11
PyQt5-Qt5 5.15.15
PyQt5_sip 12.15.0
PyQtWebEngine 5.12
pyrsistent 0.18.0
PySocks 1.7.1
pytest 6.2.4
python-dateutil 2.8.2
python-dotenv 1.0.1
python-jose 3.3.0
python-lsp-black 1.0.0
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.2.4
python-multipart 0.0.17
python-slugify 5.0.2
pytz 2021.3
PyWavelets 1.1.1
pyxdg 0.27
PyYAML 6.0
pyzmq 22.2.1
QDarkStyle 3.0.2
qstylizer 0.1.10
QtAwesome 1.0.2
qtconsole 5.1.1
QtPy 1.10.0
quantile-python 1.1
ray 2.6.3
regex 2021.8.3
requests 2.26.0
rich 13.9.4
rope 0.19.0
rpyc 6.0.0
rsa 4.9
Rtree 0.9.7
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.12
ruamel-yaml-conda 0.15.100
ruff 0.8.0
s3fs 2023.10.0
safetensors 0.4.5
scikit-image 0.18.3
scikit-learn 0.24.2
scikit-learn-intelex 2021.20210714.170444
scipy 1.7.1
seaborn 0.11.2
SecretStorage 3.3.1
selenium 4.9.1
semantic-version 2.10.0
Send2Trash 1.8.0
setproctitle 1.3.4
setuptools 58.0.4
shellingham 1.5.4
simplegeneric 0.8.1
singledispatch 3.7.0
sip 4.19.13
six 1.16.0
sniffio 1.3.1
snowballstemmer 2.1.0
sortedcollections 2.1.0
sortedcontainers 2.4.0
soupsieve 2.2.1
Sphinx 4.2.0
sphinxcontrib-applehelp 1.0.2
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport 1.2.4
spyder 5.1.5
spyder-kernels 2.1.3
SQLAlchemy 1.4.22
sse-starlette 2.1.3
starlette 0.37.2
statsmodels 0.12.2
style 1.1.0
sympy 1.9
tables 3.6.1
tabulate 0.9.0
TBB 0.2
tblib 1.7.0
tensorboardX 2.6.2.2
terminado 0.9.4
testpath 0.5.0
text-unidecode 1.3
textdistance 4.2.1
threadpoolctl 2.2.0
three-merge 0.1.1
tifffile 2021.7.2
timm 1.0.11
tinycss 0.4
tokenizers 0.20.4
toml 0.10.2
tomlkit 0.12.0
toolz 0.11.1
torch 2.1.2+cu121
torchaudio 2.1.2+cu121
torchvision 0.16.2+cu121
tornado 6.3.2
tqdm 4.67.1
traitlets 5.1.0
transformers 4.46.3
trio 0.27.0
trio-websocket 0.11.1
triton 2.1.0
typed-ast 1.4.3
typer 0.11.1
typing_extensions 4.12.2
ujson 4.0.2
unicodecsv 0.14.1
Unidecode 1.2.0
update 0.0.1
urllib3 1.26.7
uvicorn 0.32.1
uvloop 0.21.0
watchdog 2.1.3
wcwidth 0.2.5
webencodings 0.5.1
websocket-client 1.8.0
websockets 11.0.3
Werkzeug 2.0.2
wheel 0.37.0
whichcraft 0.6.1
widgetsnbextension 3.5.1
wrapt 1.12.1
wsproto 1.2.0
wurlitzer 2.1.1
xinference 0.12.0
xlrd 2.0.1
XlsxWriter 3.0.1
xlwt 1.3.0
xmltodict 0.12.0
xoscar 0.4.1
yapf 0.31.0
yarl 1.18.0
zict 2.0.0
zipp 3.6.0
zope.event 4.5.0
zope.interface 5.4.0

The command used to start Xinference / 用以启动 xinference 的命令

xinference -p 9997 -H 0.0.0.0无法正常启动，感觉是版本不匹配的问题
将xinference升级到1.0.0也报相同的错误
将pydantic降级到2.4.2还是报相同的错误
卸载xinference后重新安装0.12.0版本和1.0.0均无法解决问题，均报相同的错误

Reproduction / 复现过程

(llm) wpz@a800-04:~$ xinference -p 9997 -H 0.0.0.0pip list
Traceback (most recent call last):
File "/home/wpz/anaconda3/envs/llm/bin/xinference", line 5, in
from xinference.deploy.cmdline import cli
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/init.py", line 38, in
_install()
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/init.py", line 35, in _install
install_model()
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/model/init.py", line 17, in _install
from .llm import _install as llm_install
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/model/llm/init.py", line 20, in
from .core import (
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/model/llm/core.py", line 26, in
from ...types import PeftModelConfig
File "/home/wpz/anaconda3/lib/python3.9/site-packages/xinference/types.py", line 417, in
from openai.types.completion_create_params import CompletionCreateParamsNonStreaming
File "/home/wpz/anaconda3/lib/python3.9/site-packages/openai/init.py", line 8, in
from . import types
File "/home/wpz/anaconda3/lib/python3.9/site-packages/openai/types/init.py", line 5, in
from .batch import Batch as Batch
File "/home/wpz/anaconda3/lib/python3.9/site-packages/openai/types/batch.py", line 7, in
from .._models import BaseModel
File "/home/wpz/anaconda3/lib/python3.9/site-packages/openai/_models.py", line 85, in
class BaseModel(pydantic.BaseModel):
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 221, in new
set_model_fields(cls, bases, config_wrapper, ns_resolver)
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 593, in set_model_fields
fields, class_vars = collect_model_fields(cls, bases, config_wrapper, ns_resolver, typevars_map=typevars_map)
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_fields.py", line 112, in collect_model_fields
type_hints = _typing_extra.get_model_type_hints(cls, ns_resolver=ns_resolver)
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_typing_extra.py", line 509, in get_model_type_hints
hints[name] = try_eval_type(value, globalns, localns)
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_typing_extra.py", line 555, in try_eval_type
value = _type_convert(value)
File "/home/wpz/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_typing_extra.py", line 468, in _type_convert
return _make_forward_ref(arg, is_argument=False, is_class=True)
TypeError: init() got an unexpected keyword argument 'is_class'

Expected behavior / 期待表现

正常运行

[codingl2k1]

Simillar issue: pydantic/pydantic#6244. Which version of python are you using?

[ybsbbw]

Python3.9-------- 原始邮件 --------发件人： codingl2k1 ***@***.***>日期： 2024年11月27日周三 23:32收件人： xorbitsai/inference ***@***.***>抄送： ybsbbw ***@***.***>, Author ***@***.***>主 题： Re: [xorbitsai/inference] xinference无法正常运行 (Issue #2597) Simillar issue: pydantic/pydantic#6244. Which version of python are you using? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[948024326]

Python3.9-------- 原始邮件 --------发件人： codingl2k1 @.>日期： 2024年11月27日周三 23:32收件人： xorbitsai/inference @.>抄送： ybsbbw @.>, Author @.>主 题： Re: [xorbitsai/inference] xinference无法正常运行 (Issue #2597) Simillar issue: pydantic/pydantic#6244. Which version of python are you using? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

试下python3.10.0 同样的GPU卡 我cuda是12.4版本 没问题

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 5 days since being marked as stale.