[platforms] enable platform plugins

[youkaichao]

refactor of #11222 , enable oot-registered platforms.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[wangxiyuan]

if they import current_platform too early, then this test will fail, and the error message will tell them which line to blame.

I'm OK with this. Looks clear enough

[wangxiyuan]

This is removed directly. It's useless now?

[youkaichao]

it is moved to https://github.com/vllm-project/vllm/pull/11602/files#diff-1ca8953d4dab62fff08a34c8cf370d2a37aeb009526ab4fc5464a65e1d03b036R84 .

this line does nothing actually. no subclass calls this function.

[wangxiyuan]

LGTM

[DarkLight1337]

Correct me if I'm wrong, but isn't the main reason for lazy init is to allow platform plugins to be loaded before the built-in ones?

[DarkLight1337]

The circular import problem should already be solved by returning strings in the built-in plugins instead of direct import.

[youkaichao]

yes, you are right.

what this line of comment mean:

for third-party platform developers, they also need to import vllm.platforms and inherit the base Platform. therefore, we cannot resolve the current platform during import vllm.platforms

[DarkLight1337]

I think we can avoid this import by moving the code into a registry.py, similar to that for models.

[youkaichao]

even if we have vllm.platforms.registry , when people import that module, vllm/platforms/__init__.py will be executed.

[DarkLight1337]

We can also move current_platform into vllm.platforms.registry so it's not imported automatically when accessing other platform modules.

[youkaichao]

explained in 8f43a03 . let me know if it works.

[DarkLight1337]

This is much clearer now, thanks!

[DarkLight1337]

I really don't like the necessity of the lazy import (similar to CUDA re-initialization problem) but we can iterate on this later if needed...

[youkaichao]

I really don't like the necessity of the lazy import (similar to CUDA re-initialization problem)

I agree we should avoid lazy import if possible. But there are some historical code we need to take care of, many people unconsciously initialize cuda or trigger current_platform resolution :(

[MengqingCao]

Sorry for the delay, Overall lgtm.

Just one question. Maybe this pr lack of a func like is_oot_platform() in vllm/platforms/interface.py? I think we should expose a similar api for oot platforms to use. And an expanded PlatformEnum is also needed for this.

[youkaichao]

Maybe this pr lack of a func like is_oot_platform() in vllm/platforms/interface.py? I think we should expose a similar api for oot platforms to use. And an expanded PlatformEnum is also needed for this.

We can add it later if it is necessary. Right now I don't see the necessity. For the oot platform related code, they always run on oot platforms; for vllm's main branch code, they don't need to consider oot platform code.

I think vllm mainly exposes integration hooks to oot platforms. for example, when vllm's code use current_platform.device_name , oot platform should directly benefit from it by providing its own device_name . I don't see when it is necessary to tell whether we are using a builtin platform or oot platform.

[MengqingCao]

We can add it later if it is necessary. Right now I don't see the necessity. For the oot platform related code, they always run on oot platforms; for vllm's main branch code, they don't need to consider oot platform code.

I think vllm mainly exposes integration hooks to oot platforms. for example, when vllm's code use current_platform.device_name , oot platform should directly benefit from it by providing its own device_name . I don't see when it is necessary to tell whether we are using a builtin platform or oot platform.

OK, thx! We will verify NPU as an oot device recently. If necessary, I will add it through a new PR.

[youkaichao]

failed tests are not related, merging.