Installation fails due to TensorRT 8.6.1 compatibility issues

[nguyenphuvinhtoan]

Description

When trying to create the conda environment using environment.yaml, the installation fails due to TensorRT dependency issues. Specifically, the following errors occur:

1. Unable to find compatible tensorrt-libs==8.6.1

   • Only newer versions (9.x.x) are available in the package repository
   • Available versions start from 9.0.0.post11.dev1 and up

2. Python version compatibility warnings for various dependencies

Error Log

ERROR: Could not find a version that satisfies the requirement tensorrt-libs==8.6.1
(from versions: 9.0.0.post11.dev1, 9.0.0.post12.dev1, 9.0.1.post11.dev4, 9.0.1.post12.dev4, 9.1.0.post11.dev4, etc.)
ERROR: No matching distribution found for tensorrt-libs==8.6.1

Environment Details

• OS: Linux (Ubuntu)
• Python version: 3.10
• Installation method: conda env create -f environment.yaml

Questions

1. Is there an alternative source for TensorRT 8.6.1?
2. Can you provide a detailed instruction for installation?

Additional Context

The installation process successfully downloads and prepares most dependencies but fails specifically at the TensorRT installation step. This appears to be due to version availability in the current package repositories.

Image error

[digital-avatar]

@nguyenphuvinhtoan

Can you show your GPU, driver version, and CUDA version?

According to my experience, the model is not very sensitive to the version of the dependent library, as long as tensorrt can be imported normally. So you can also try to install the corresponding library according to the actual situation, and then re-execute the tensorRT model conversion script (script/cvt_onnx_to_trt.py).

[nguyenphuvinhtoan]

Here is my GPU and CUDA version:

I got this error

I have faced with this error when run this below code to create conda environment:

conda env create -f environment.yaml

[digital-avatar]

@nguyenphuvinhtoan
Sorry, I don't have a T4 environment, so I can't verify it directly. I plan to deploy a demo in the T4 environment of colab, but I haven't had time to implement it yet.

The possible reason for the environment problem is that the pytorch version in environment.yaml is incompatible with your environment. You can install the following libraries based on any version of pytorch in cuda12 to see if it can be installed normally.
pip install numpy==2.0.1 tensorrt==8.6.1 librosa tqdm filetype imageio opencv_python_headless scikit-image cython cuda-python imageio-ffmpeg colored polygraphy

In addition, you can confirm whether you have manually configured the pip and conda sources. If you still cannot find the 8.x version of tensorRT, you can try to install other versions, such as 9.x.

[nitinmukesh]

Sam error, tensorrt==8.6.1 not found.
if I install higher version it starts to give another error. I searched the missing DLL and it is present. cuDNN is also installed.
I asked for pip freeze here #4 (comment)

[nitinmukesh]

9.x is also not supported, so only 10.x which gives above error.
https://pypi.org/project/tensorrt/#history

[huangzhike]

pls try install from nvidia source

[nguyenphuvinhtoan]

Successfully Set Up the Environment with Tesla T4

I’ve successfully set up my environment to work with Tesla T4. Below are the steps I followed:

1. Firstly, I do a clean install cuda-toolkit-11.8 from source

2. Secondly, I install cuDNN 8.9.0

• Download page: https://developer.nvidia.com/rdp/cudnn-archive (CUDA 11.x)
• Here is the tutorial for install cuDNN (maybe it can help in some case) https://gist.github.com/denguir/b21aa66ae7fb1089655dd9de8351a202

3. Thirdly, I install tensorRT 8.6.1.6 from source: https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-861/install-guide/index.html

4. Finally, I install tensorrt-libs 8.6.1 from https://pypi.nvidia.com/ (You can refer this link to find out the best match with your cuda version)

pip install --no-cache-dir --extra-index-url https://pypi.nvidia.com tensorrt-cu11-libs==8.6.1

Requirements File

Below is the requirements.txt file that works well with Tesla T4: requirements.txt

If you not have GPU with Ampere architecture, remember to convert ONNX to TensorRT follow the ❗Note part of author

[digital-avatar]

@nguyenphuvinhtoan Great, thanks for sharing your solution.

[digital-avatar]

@nitinmukesh
For Windows solutions, please refer to this implementation of @justinjohn0306.

[BlueSkyyyyyy]

try to use the docker images nvcr.io/nvidia/tensorrt:24.01-py3 and pip install requirements as
torch audioread==3.0.1 cffi==1.17.1 cuda-python==12.6.2.post1 cython==3.0.11 decorator==5.1.1 filetype==1.2.0 imageio==2.36.1 imageio-ffmpeg==0.5.1 joblib==1.4.2 lazy-loader==0.4 librosa==0.10.2.post1 llvmlite==0.43.0 msgpack==1.1.0 numba==0.60.0 opencv-python-headless==4.10.0.84 packaging==24.2 platformdirs==4.3.6 pooch==1.8.2 pycparser==2.22 scikit-image==0.25.0 scikit-learn==1.6.0 scipy==1.15.0 soundfile==0.13.0 soxr==0.5.0.post1 threadpoolctl==3.5.0 tifffile==2024.12.12 tqdm==4.67.1
it worked for me to run inference.py example as so far.

[WeizhenEricFang]

Did Anyone get this error when importing the tensorrt ?

`>>> import tensorrt
Traceback (most recent call last):
File "", line 1, in
File "/home/jovyan/.conda/envs/ditto/lib/python3.10/site-packages/tensorrt/init.py", line 18, in
from tensorrt_bindings import *
File "/home/jovyan/.conda/envs/ditto/lib/python3.10/site-packages/tensorrt_bindings/init.py", line 67, in
from .tensorrt import *

ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory`

[nguyenphuvinhtoan]

@WeizhenEricFang
This error may occur if cuDNN is not installed correctly. Please verify that cuDNN has been installed properly. It is recommended to use cuDNN versions 8.9.0 to ensure TensorRT 8.6.1 functions correctly.
The installation cuDNN here: https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-861/install-guide/index.html