Docker-Slim Error - NVIDIA Container Initialization Failure

[Galois123]

Description

When using slim build to minimize an image that includes NVIDIA GPU support, the following error occurred. The error log indicates that the NVIDIA container runtime failed to recognize the specified GPU device ID during initialization, causing the container startup to fail.

Error Log

Below is the full error log:

Time="2025-02-25T11:23:25+08:00" level=error msg=terminating error="API error (500): OCI runtime create failed: container_linux.go:265: starting container process caused \\\"process_linux.go:368: container init caused \\\"process_linux.go:351: running prestart hook 1 caused \\\\\\\"error running hook: exit status 1, stdout:, stderr: exec command: [/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig --device=GPU-8ae4147a-9bb4-5132-e109-a557ddf32e51,GPU-fied2ebc-66f8-9bb0-426e-8fea176701f6 --compute --utility--require=cuda>=10.1 brand=tesla,driver>=396,driver<397 brand-tesla,driver>-410,driver<411 brand-tesla,driver>=418,driver<419 --pid=38463 /docker-data/docker/overlay2/861e0c9bede91cb70a6f52fca360c211dcb0a4e605b3f83de8325c7f44c6bc2a/merged]\\\\\\\\nnvidia-container-cli: device error: unknown device id: GPU-8ae4147a-9bb4-5132-e109-a557ddf32e51\\\\\\\n\\\\\\\\"\\\": unknown"stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x5e\ngithub.com/slimtoolkit/slim/pkg/app.(*ExecutionContext).FailOn(0xc0006d0c40, {0x23298a0, 0xc0007d4558})\n\tgithub.com/slimtoolkit/slim/pkg/app/execontext.go:58 +0x58\ngithub.com/slimtoolkit/slim/pkg/app/master/command/build.OnCommand(_, _, _, _, _, _)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/build/handler.go:1136 +0x46a5\ngithub.com/slimtoolkit/slim/pkg/app/master/command/build.glob..func1(0xc0004e5f80)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/build/cli.go:760 +0x5d51\ngithub.com/urfave/cli/v2.(*Command).Run(0x33a7220, 0xc0004e5f80, {0xc0004e5fc0, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x9dd\ngithub.com/urfave/cli/v2.(*Command).Run(0xc00056a580, 0xc0004e59c0, {0xc0001b4000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0xc2e\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc0007b4a00, {0x23470c8, 0x341e0a0}, {0xc0001b4000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x5db\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15 +0x45\nmain.main()\n\tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15 +0x187\n"