genai version is conflict with openvino

[azhai219]

1. install genai
2. uninstall openvino
3. build the latest openvino. Latest version is 25.0

when try to operate benchmark.py, the error is like:

python benchmark.py -m ... -p ../prompts.json -n 3 -d CPU
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
Traceback (most recent call last):
  File "/home/x/llm_dev/openvino.genai/tools/llm_bench/benchmark.py", line 15, in <module>
    import task.visual_language_generation as bench_vlm
  File "/home/x/llm_dev/openvino.genai/tools/llm_bench/task/visual_language_generation.py", line 9, in <module>
    import llm_bench_utils.ov_utils
  File "/home/x/llm_dev/openvino.genai/tools/llm_bench/llm_bench_utils/ov_utils.py", line 22, in <module>
    import openvino_genai as ov_genai
  File "/home/x/llm_dev/venv/llm/lib/python3.10/site-packages/openvino_genai/__init__.py", line 14, in <module>
    from .py_openvino_genai import (
ImportError: libopenvino.so.2460: cannot open shared object file: No such file or directory

[as-suvorov]

Could you please try to install latest packages from nightly:

pip install -U --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly openvino_tokenizers openvino openvino-genai

[azhai219]

@as-suvorov Hi, your command can resolve this issue. Thanks. Could you add this bug WA to someplace, which can help others.

[azhai219]

reopen the issue. @avinash-palleti Could you help take a look? Thanks.

python openvino.genai/tools/llm_bench/benchmark.py -m /mnt/disk1/x/baichuan2-13b-chat/pytorch/ov/FP16 -d cpu -n 1 -mc 1 -pf baichuan2-13b-chat.jsonl -ic 32
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
Traceback (most recent call last):
  File "/home/xx/workspace/llm_dev/openvino.genai/tools/llm_bench/benchmark.py", line 15, in <module>
    import task.visual_language_generation as bench_vlm
  File "/home/x/workspace/llm_dev/openvino.genai/tools/llm_bench/task/visual_language_generation.py", line 9, in <module>
    import llm_bench_utils.ov_utils
  File "/home/x/workspace/llm_dev/openvino.genai/tools/llm_bench/llm_bench_utils/ov_utils.py", line 22, in <module>
    import openvino_genai as ov_genai
  File "/home/x/workspace/llm_dev/venv/llm/lib/python3.12/site-packages/openvino_genai/__init__.py", line 14, in <module>
    from .py_openvino_genai import (
ImportError: /home/x/workspace/llm_dev/venv/llm/lib/python3.12/site-packages/openvino_genai/libopenvino_genai.so.2500: undefined symbol: _ZN2ov3Any4Base9to_stringEv

[ilya-lavrenov]

install genai
uninstall openvino
build the latest openvino. Latest version is 25.0

the order of steps is invalid. Let's see what happened:

1. You have install GenAI and it builds against OpenVINO downloaded from pypi or https://storage.openvinotoolkit.org/simple/wheels/nightly

   openvino.genai/pyproject.toml

   Lines 52 to 58 in 09a5426

   	[build-system]
   	requires = [
   	"py-build-cmake==0.3.4",
   	"openvino~=2025.0.0.0.dev",
   	"pybind11-stubgen==2.5.1",
   	"cmake~=3.23.0"
   	]

   Since OpenVINO on pypi has tag manylinux_2014, it means it was built on CentOS 7 with old gcc ABI => GenAI is compiled with the same ABI
2. You have removed OpenVINO built for CentOS
3. You have built and installed OpenVINO on your local machine. It has new ABI, which is incompatible with GenAI built on step 1

So, the correct steps would be install base dependency first and then build other packages on top of it. E.g.

• Uee pip install -U --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly openvino_tokenizers openvino openvino-genai provided above, which ensures latest versions of OpenVINO and GenAI
• If you want to build everything by yourself, then build OpenVINO first, then build Tokenizers and GenAI:

git clone https://github.com/openvinotoolkit/openvino.git
cmake -S openvino -B build_folder
cmake --build build_folder --parallel
pip3 install openvino --find-links=build_folder/wheels
pip3 install git+https://github.com/openvinotoolkit/openvino_tokenizers.git --find-links=build_folder/wheels
pip3 install git+https://github.com/openvinotoolkit/openvino.genai.git --find-links=build_folder/wheels

Note, that --find-links in Tokenizers / GenAI build is required to ensure that Tokenizers and GenAI are build against ABI of your locally built OpenVINO.