Using docker build makes torch cuda is_available to False!

[innat]

Describe the bug

I've GPU attached in my system. I tried to run Dockerfile, and expected that the image will be built with GPU support. But it isn't.

root@00921839c781:/anomalib# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_Oct_11_21:27:02_PDT_2021
Cuda compilation tools, release 11.4, V11.4.152
Build cuda_11.4.r11.4/compiler.30521435_0

root@00921839c781:/anomalib# nvidia-smi
bash: nvidia-smi: command not found

root@00921839c781:/anomalib# python
>>> import torch
>>> torch.cuda.is_available()
>>> Flase

To Reproduce

1. build image: docker build -t test .
2. run it: docker run -it test

Expected behaviour

As it's based on nvidia-cuda, I expected it would work out of the box if GPU is attached in the system.

Hardware and Software Configuration

• OS: [Ubuntu, OD]
• NVIDIA Driver Version [470.57.02]
• CUDA Version [e.g. 11.4]
• CUDNN Version [e.g. v11.4.120]
• OpenVINO Version [Optional e.g. v2021.4.2]

Additional context

• https://discuss.pytorch.org/t/userwarning-cuda-initialization-the-nvidia-driver-on-your-system-is-too-old-found-version-10010/141547

[samet-akcay]

@ashwinvaidya17, will you be able to have a look at this?

[ashwinvaidya17]

@innat have you tried passing --gpus=all to docker run command? You can read more here: https://openvinotoolkit.github.io/anomalib/guides/developing_on_docker.html#run-the-docker-image?

[innat]

Yes, I tried that already. I tried to run this docker file in GCP ; (Architecture: x86_64, Docker version 20.10.18, build b40c2f6). GPU is attached and necessary drived is installed (auto). Is there anything else need to care manually regarding NVIDIA Container Toolkit? In your env, is it working (cuda.is_available() -> True ? inside the docker?)

[ashwinvaidya17]

That's weird. I just build a new image and it is working. From your screenshot I can see that you passed the gpu flag at the end. Try passing it before the -it flag.