Platform plugins and current_platform

Code version: vLLM @ 3e44c0d · 2026-10-22

The idea#

vLLM runs on NVIDIA and AMD GPUs, on CPUs, on Intel XPUs and on other hardware. Much of its code must ask: which device do I run on, which attention backend can I use, which communicator do I use for collectives? vLLM puts all these answers into one object, the platform. Code reads it as current_platform. Its class is a subclass of Platform, for example CudaPlatform.

vLLM must choose the class before it can use it. A platform detector is a small function that checks for one kind of hardware. It returns the dotted name of a Platform subclass when the hardware is there, and None when it is not. vLLM has five built-in detectors. A hardware vendor adds a sixth one as a plugin in the group vllm.platform_plugins. When exactly one out-of-tree detector answers, it wins over the built-in ones.

The choice happens late, on the first read of current_platform, not when Python imports vllm.platforms. The reason: an out-of-tree platform must import the Platform base class from vLLM before vLLM chooses a platform. Each process of vllm serve makes this choice again, because each process has its own Python objects.

An everyday comparison. Think of a universal remote control. The first time that you press a button, it tests which television is in the room. Then it uses the codes for that television for as long as it runs. A vendor can add a code card for a new television, and a code card always wins over the built-in codes. The comparison stops here. If two code cards answer, vLLM stops with an error. And each process of vLLM is a separate remote control that tests the room again.

The picture#

Read panel 1 from left to right: this is the code path of one process. Panel 2 is the decision after all detectors have run. Note the order: the code looks at the out-of-tree detectors first, so the built-in count does not matter when one out-of-tree detector is active.

In the code#

Step 1: current_platform is made on first use#

vllm/platforms/__init__.py has no variable named current_platform. It has a module-level __getattr__ function. Since Python 3.7 (PEP 562), Python calls such a function when code reads a name that the module does not have:

_current_platform = None
_init_trace: str = ""

if TYPE_CHECKING:
    current_platform: Platform


def __getattr__(name: str):
    if name == "current_platform":
        # lazy init current_platform.
        # 1. out-of-tree platform plugins need `from vllm.platforms import
        #    Platform` so that they can inherit `Platform` class. Therefore,
        #    we cannot resolve `current_platform` during the import of
        #    `vllm.platforms`.
        # 2. when users use out-of-tree platform plugins, they might run
        #    `import vllm`, some vllm internal code might access
        #    `current_platform` during the import, and we need to make sure
        #    `current_platform` is only resolved after the plugins are loaded
        #    (we have tests for this, if any developer violate this, they will
        #    see the test failures).
        global _current_platform
        if _current_platform is None:
            platform_cls_qualname = resolve_current_platform_cls_qualname()
            _current_platform = resolve_obj_by_qualname(platform_cls_qualname)()
            global _init_trace
            _init_trace = "".join(traceback.format_stack())
        return _current_platform
    elif name in globals():
        return globals()[name]
    else:
        raise AttributeError(f"No attribute named '{name}' exists in {__name__}.")

1. Line 296. _current_platform starts as None in each process.
2. Lines 299-300. The name exists only for type checkers. At run time it is not a module attribute, so the import from vllm.platforms import current_platform calls __getattr__.
3. Line 318. The detection returns a dotted class name, for example "vllm.platforms.cuda.CudaPlatform".
4. Line 319. resolve_obj_by_qualname imports the module and gets the class (vllm/utils/import_utils.py:119-123). The () makes the one object.
5. Lines 320-321. _init_trace keeps the stack of the first read. A test prints it when a platform resolves too early (tests/plugins_tests/test_platform_plugins.py:28-34).

The comment gives the reason for the late choice. An out-of-tree platform needs from vllm.platforms import Platform to define its subclass. If the import of vllm.platforms chose the platform, the plugin could not load first. A test checks that import vllm alone does not resolve the platform (tests/plugins_tests/test_platform_plugins.py:37-48).

Step 2: resolve_current_platform_cls_qualname, part 1#

The first part handles the CPU target and runs every detector:

def resolve_current_platform_cls_qualname() -> str:
    # An explicit CPU target is authoritative. Native CPU-only CI jobs reuse
    # an accelerator wheel and can run on accelerator hosts, so probing every
    # plugin would otherwise activate both CPU and the host accelerator.
    if envs.VLLM_TARGET_DEVICE == "cpu":
        cpu_platform_cls_qualname = cpu_platform_plugin()
        assert cpu_platform_cls_qualname is not None
        logger.debug("Explicitly selected CPU platform.")
        return cpu_platform_cls_qualname

    platform_plugins = load_plugins_by_group(PLATFORM_PLUGINS_GROUP)

    activated_plugins = []

    for name, func in chain(builtin_platform_plugins.items(), platform_plugins.items()):
        try:
            assert callable(func)
            platform_cls_qualname = func()
            if platform_cls_qualname is not None:
                activated_plugins.append(name)
        except Exception:
            logger.debug(
                "Platform plugin %s failed during detection.",
                name,
                exc_info=True,
            )

1. Lines 244-248. If VLLM_TARGET_DEVICE is cpu, the function returns the CPU class at once. It does not load the plugin group and does not run any other detector. The comment says why: CPU-only test jobs run on hosts with accelerators. The default value is cuda (vllm/envs.py:622).
2. Line 250. load_plugins_by_group imports the detectors of vllm.platform_plugins. VLLM_PLUGINS filters them, as for the other groups (vllm/plugins/__init__.py:63-64).
3. Line 254. The loop runs the five built-in detectors first, then the out-of-tree ones.
4. Lines 257-259. A detector is active when it returns a name, not None.
5. Lines 260-265. An exception in a detector is logged at DEBUG level, and the detector counts as not active.

Step 3: resolve_current_platform_cls_qualname, part 2#

    activated_builtin_plugins = list(
        set(activated_plugins) & set(builtin_platform_plugins.keys())
    )
    activated_oot_plugins = list(set(activated_plugins) & set(platform_plugins.keys()))

    if len(activated_oot_plugins) >= 2:
        raise RuntimeError(
            "Only one platform plugin can be activated, but got: "
            f"{activated_oot_plugins}"
        )
    elif len(activated_oot_plugins) == 1:
        platform_cls_qualname = platform_plugins[activated_oot_plugins[0]]()
        logger.info("Platform plugin %s is activated", activated_oot_plugins[0])
    elif len(activated_builtin_plugins) >= 2:
        raise RuntimeError(
            "Only one platform plugin can be activated, but got: "
            f"{activated_builtin_plugins}"
        )
    elif len(activated_builtin_plugins) == 1:
        platform_cls_qualname = builtin_platform_plugins[activated_builtin_plugins[0]]()
        logger.debug(
            "Automatically detected platform %s.", activated_builtin_plugins[0]
        )
    else:
        platform_cls_qualname = "vllm.platforms.interface.UnspecifiedPlatform"
        logger.debug("No platform detected, vLLM is running on UnspecifiedPlatform")
    return platform_cls_qualname

1. Lines 267-270. The active names are split into built-in and out-of-tree. The split uses sets, so the order of the names in the error text can vary.
2. Lines 272-276. Two or more active out-of-tree detectors: RuntimeError.
3. Lines 277-279. Exactly one: vLLM calls the detector again to get the name, and logs "Platform plugin ... is activated" at INFO level. The built-in detectors do not matter here.
4. Lines 280-284. No out-of-tree detector and two or more built-in ones: RuntimeError.
5. Lines 285-289. Exactly one built-in detector: it is called again, with a DEBUG log.
6. Lines 290-292. No active detector: UnspecifiedPlatform, with a DEBUG log only.

UnspecifiedPlatform has device_type = "" (vllm/platforms/interface.py:1493-1495). Most methods of the base class raise NotImplementedError or return a default, so start-up fails later with a less clear error.

Step 4: the five built-in detectors#

The dict builtin_platform_plugins maps a name to each detector (vllm/platforms/__init__.py:231-237). Each detector checks for its hardware without starting a device context:

Even the built-in TPU platform lives in another package. vllm/platforms/tpu.py only imports TpuPlatform from tpu_inference, and logs an error if that package is missing (vllm/platforms/tpu.py:9-20).

Step 5: the platform comes before the general plugins#

In every process of vllm serve, the platform resolves before the first general plugin runs. A module-level from vllm.platforms import current_platform in the import chain triggers __getattr__ early, for example in vllm/engine/arg_utils.py (vllm/engine/arg_utils.py:122) and in vllm/config/compilation.py (vllm/config/compilation.py:22). The page How vLLM finds and loads plugins traces the chain for the front end, the engine core and the workers. So a general plugin cannot change which platform vLLM detects. A platform package that needs general setup publishes a second entry point (Step 6).

Step 6: what a platform plugin package looks like#

vLLM's test platform is the smallest complete example. Its setup.py publishes two entry points:

setup(
    name="vllm_add_dummy_platform",
    version="0.1",
    packages=["vllm_add_dummy_platform"],
    entry_points={
        "vllm.platform_plugins": [
            "dummy_platform_plugin = vllm_add_dummy_platform:dummy_platform_plugin"  # noqa
        ],
        "vllm.general_plugins": [
            "dummy_custom_ops = vllm_add_dummy_platform:register_ops"
        ],
    },
)

The detector returns the dotted name of the class. This test detector always answers, because it runs only in a test environment:

def dummy_platform_plugin() -> str | None:
    # Real platform plugins often import vLLM modules while registering.
    from vllm.utils.torch_utils import direct_register_custom_op  # noqa: F401

    return "vllm_add_dummy_platform.dummy_platform.DummyPlatform"


def register_ops():
    import vllm_add_dummy_platform.dummy_custom_ops  # noqa

The class sets the identity fields and overrides two hooks. _enum = PlatformEnum.OOT makes is_out_of_tree() true (vllm/platforms/interface.py:218-219):

class DummyPlatform(Platform):
    _enum = PlatformEnum.OOT
    device_name = "DummyDevice"
    device_type: str = "privateuseone"
    dispatch_key: str = "PrivateUse1"

    @classmethod
    def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
        vllm_config.compilation_config.custom_ops = ["all"]

    @classmethod
    def get_attn_backend_cls(
        cls,
        selected_backend: "AttentionBackendEnum",
        attn_selector_config: "AttentionSelectorConfig",
        num_heads: int | None = None,
    ) -> str:
        return "vllm_add_dummy_platform.dummy_attention_backend.DummyAttentionBackend"  # noqa E501

A real detector checks for its device and returns None when the device is missing. The design doc says the same (docs/design/plugin_system.md:50). The page The Platform interface explains which methods a real platform must override, for example check_and_update_config, which must set worker_cls.

Step 7: out-of-tree platforms named in the repository#

• tpu_inference. The TPU detector returns a class from the external tpu_inference package when JAX_PLATFORMS contains "proxy" (Pathways) (vllm/platforms/__init__.py:36-39, vllm/envs.py:1594-1596). Without Pathways, vllm/platforms/tpu.py re-exports the class of that package (vllm/platforms/tpu.py:9-14).
• Other hardware plugins. The docs list "Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more" as hardware plugins (docs/README.md:64). The code of these plugins is not in this repository, so this site does not describe them.

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

What can go wrong#

• Two platform plugins are installed. If both detectors answer, start-up stops with "Only one platform plugin can be activated" (vllm/platforms/__init__.py:272-276). Uninstall one, or make its detector return None.
• Your detector raises. vLLM logs it only at DEBUG level and uses another platform (vllm/platforms/__init__.py:260-265). Run with VLLM_LOGGING_LEVEL=DEBUG to see the traceback.
• The platform resolves too early. If code reads current_platform during import vllm, the detection runs the platform plugins against a half-imported vLLM. A test checks this (tests/plugins_tests/test_platform_plugins.py:37-48). Another test prints _init_trace to find the first read (tests/plugins_tests/test_platform_plugins.py:28-34).
• No platform is found. UnspecifiedPlatform has an empty device_type (vllm/platforms/interface.py:1493-1495). The first device call then fails far from the cause.
• libtpu without tpu_inference. The TPU detector answers vllm.platforms.tpu.TpuPlatform, but that module defines the name only when tpu_inference imports (vllm/platforms/tpu.py:9-20). The import of the class then fails. (Derived.)

Why it matters. The platform decides the attention backend, the communicator, the worker class and many defaults. A wrong platform gives errors that look like model or kernel bugs. Check the platform first: look for the "activated" line, or log at DEBUG level.