[ci][misc] fix more device count

[youkaichao]

No description provided.

[DarkLight1337]

Since you got some experience already, could you provide a list of which third-party imports may cause CUDA initialization so we can better avoid importing them too early? It would help a lot when adding new distributed tests.

[youkaichao]

I'm writing a debug guide for this.

[DarkLight1337]

Are you going to fix other occurrences of torch.cuda.device_count in this PR? I see that it is called in some other tests as well.

[youkaichao]

@DarkLight1337 this pr fixes implicit call to torch.cuda.device_count() . Those explicit call can be removed later in a followup PR.

[DarkLight1337]

I found that using lazy import in vllm.transformer_utils.image_processor seems to remove the need for lazy importing the multimodal modules. See the changes to that file in #5276.

[youkaichao]

You can see my stack trace at #6056 (comment) . Yes, image_processor is the part to blame. Lazy import either one would work.

[DarkLight1337]

I found that using lazy import in vllm.transformer_utils.image_processor seems to remove the need for lazy importing the multimodal modules. See the changes to that file in #5276.

Let's do it this way since vllm.transformer_utils.image_processor is the source of the problem and is imported in fewer places.

[youkaichao]

cool, feel free to submit another pr to suppress this one.

[DarkLight1337]

#5276 is getting pretty close to merge so I'm inclined to just use that PR. Is that ok with you?

[youkaichao]

close as #5276 will include the fix