OSError: Can't load tokenizer for 'bert-base-uncased'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'bert-base-uncased' is the correct path to a directory containing all relevant files for a BertTokenizer tokenizer.

[Shimooth]

I encountered an error while running the playground.py
Can you help me check where the problem is? Thank you very much!

I have already set up the environment according to the steps.

The files needed for download are also prepared.
vicuna-7b

Checkpoint

The file path has also been configured.

pip list:
accelerate 0.27.2
aiofiles 23.2.1
aiohttp 3.8.4
aiosignal 1.3.1
altair 5.2.0
antlr4-python3-runtime 4.9.3
anyio 4.3.0
appdirs 1.4.4
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
async-timeout 4.0.2
attrs 22.2.0
Babel 2.14.0
beautifulsoup4 4.12.3
bleach 6.1.0
blis 0.7.11
braceexpand 0.1.7
catalogue 2.0.10
cchardet 2.1.7
certifi 2024.2.2
cffi 1.16.0
chardet 5.1.0
charset-normalizer 3.3.2
click 8.1.7
comm 0.2.1
confection 0.1.4
contourpy 1.0.7
cycler 0.11.0
cymem 2.0.8
debugpy 1.8.1
decorator 5.1.1
decord 0.6.0
defusedxml 0.7.1
diffusers 0.21.4
docker-pycreds 0.4.0
exceptiongroup 1.2.0
executing 2.0.1
fastapi 0.109.2
fastjsonschema 2.19.1
ffmpy 0.3.2
filelock 3.9.0
fonttools 4.38.0
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2024.2.0
ftfy 6.1.3
gitdb 4.0.11
GitPython 3.1.42
gradio 3.24.1
gradio_client 0.0.8
h11 0.14.0
httpcore 1.0.3
httpx 0.26.0
huggingface-hub 0.20.3
idna 3.6
importlib-metadata 7.0.1
importlib-resources 5.12.0
iopath 0.1.10
ipykernel 6.29.2
ipython 8.18.1
isoduration 20.11.0
jedi 0.19.1
Jinja2 3.1.3
joblib 1.3.2
json5 0.9.17
jsonpointer 2.4
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter_client 8.6.0
jupyter_core 5.7.1
jupyter-events 0.9.0
jupyter-lsp 2.2.2
jupyter_server 2.12.5
jupyter_server_terminals 0.5.2
jupyterlab 4.1.2
jupyterlab_pygments 0.3.0
jupyterlab_server 2.25.3
kiwisolver 1.4.4
langcodes 3.3.0
lightning 2.2.0.post0
lightning-utilities 0.10.1
linkify-it-py 2.0.3
llvmlite 0.42.0
markdown-it-py 2.2.0
MarkupSafe 2.1.5
matplotlib 3.7.0
matplotlib-inline 0.1.6
mdit-py-plugins 0.3.3
mdurl 0.1.2
mistune 3.0.2
mpmath 1.3.0
multidict 6.0.4
murmurhash 1.0.10
nbclient 0.9.0
nbconvert 7.16.1
nbformat 5.9.2
nest-asyncio 1.6.0
networkx 3.2.1
nltk 3.8.1
notebook 7.1.0
notebook_shim 0.2.4
numba 0.59.0
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.19.3
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
open-clip-torch 2.24.0
opencv-python 4.9.0.80
orjson 3.9.14
overrides 7.7.0
packaging 23.0
pandas 2.2.0
pandocfilters 1.5.1
parso 0.8.3
pathlib_abc 0.1.1
pathy 0.11.0
peft 0.8.2
pexpect 4.9.0
pillow 10.2.0
pip 23.3.1
platformdirs 4.2.0
portalocker 2.8.2
preshed 3.0.9
prometheus_client 0.20.0
prompt-toolkit 3.0.43
protobuf 4.25.3
psutil 5.9.4
ptyprocess 0.7.0
pure-eval 0.2.2
pycocoevalcap 1.2
pycocotools 2.0.6
pycparser 2.21
pydantic 1.10.14
pydub 0.25.1
Pygments 2.17.2
pynndescent 0.5.11
pyparsing 3.0.9
python-dateutil 2.8.2
python-json-logger 2.0.7
python-multipart 0.0.9
pytorch-fid 0.3.0
pytorch-lightning 2.2.0.post0
pytz 2024.1
PyYAML 6.0
pyzmq 25.1.2
referencing 0.33.0
regex 2022.10.31
requests 2.31.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rouge 1.0.1
rpds-py 0.18.0
safetensors 0.4.2
scikit-learn 1.4.1.post1
scipy 1.12.0
semantic-version 2.10.0
Send2Trash 1.8.2
sentence-transformers 2.2.2
sentencepiece 0.2.0
sentry-sdk 1.40.5
setproctitle 1.3.3
setuptools 68.2.2
six 1.16.0
smart-open 6.4.0
smmap 5.0.1
sniffio 1.3.0
soupsieve 2.5
spacy 3.5.1
spacy-legacy 3.0.12
spacy-loggers 1.0.5
srsly 2.4.8
stack-data 0.6.3
starlette 0.36.3
sympy 1.12
tenacity 8.2.2
terminado 0.18.0
thinc 8.1.12
threadpoolctl 3.3.0
timm 0.6.13
tinycss2 1.2.1
tokenizers 0.13.3
tomli 2.0.1
toolz 0.12.1
torch 2.2.0
torch-fidelity 0.3.0
torchmetrics 1.3.1
torchvision 0.17.0
tornado 6.4
tqdm 4.64.1
traitlets 5.14.1
transformers 4.31.0
triton 2.2.0
typer 0.7.0
types-python-dateutil 2.8.19.20240106
typing_extensions 4.9.0
tzdata 2024.1
uc-micro-py 1.0.3
umap-learn 0.5.5
uri-template 1.3.0
urllib3 2.2.1
uvicorn 0.27.1
wandb 0.16.3
wasabi 1.1.2
wcwidth 0.2.13
webcolors 1.13
webdataset 0.2.48
webencodings 0.5.1
websocket-client 1.7.0
websockets 12.0
wheel 0.41.2
xformers 0.0.24
yarl 1.8.2
zipp 3.14.0

[KzZheng]

There should be some path issues. You can check the same issue mentioned here huggingface/transformers#16618
One potential solution is to use the absolute path.

[Shimooth]

There should be some path issues. You can check the same issue mentioned here huggingface/transformers#16618 One potential solution is to use the absolute path.

Is the path and command I configured correct?

[KzZheng]

This issue should not be related to either the vicuna model or minigpt-4 model. It is an error from Transformers lib. Bert is loaded before them.

[Shimooth]

This issue should not be related to either the vicuna model or minigpt-4 model. It is an error from Transformers lib. Bert is loaded before them.

Does this mean I need to download a bert-base-uncased model as well?

[Shimooth]

I manually downloaded the bert-base-uncased model and configured the model path in the code, which solved the previous problem, but now a 'Segmentation fault (core dumped)' error has occurred.

@KzZheng

[Shimooth]

I manually downloaded the bert-base-uncased model and configured the model path in the code, which solved the previous problem, but now a 'Segmentation fault (core dumped)' error has occurred. @KzZheng

I found that the error occurred in a line of code in the model.py script. Could you please help me take a look? @KzZheng

[KzZheng]

It seems an error from the stable diffusion pipeline. I did not encounter this error before. Maybe you should check whether you can run this pipeline first.