vLLM Explained

platforms/__init__.py: platform detection and current_platform

IntermediateCode mapabout 8 min read + 4 min deep dives

Code version: vLLM @ 3e44c0d · 2026-10-22

Purpose#

vllm/platforms/__init__.py decides which Platform subclass a process uses, and gives the one object of that class to the rest of vLLM as current_platform.

Filevllm/platforms/__init__.py, 346 lines of Python. Ten functions (five of them detectors), the dict builtin_platform_plugins and two module globals.
Main functionsresolve_current_platform_cls_qualname (vllm/platforms/__init__.py:240), __getattr__ (vllm/platforms/__init__.py:303), and the detectors tpu_platform_plugin, cuda_platform_plugin, rocm_platform_plugin, xpu_platform_plugin, cpu_platform_plugin. There is no class: the classes are in interface.py, cuda.py, rocm.py and the other files of the folder.
Runs inEvery Python process of vLLM, on the CPU, in the thread of the first reader of current_platform. With vllm serve: the front end, the EngineCore process, each worker process and the model registry subprocess.
Called byAny module that reads current_platform: 584 files in vllm/ use the name (from git grep at this commit). The first such read in a process runs the detection.
Callsload_plugins_by_group for vllm.platform_plugins (vllm/platforms/__init__.py:10), resolve_obj_by_qualname (vllm/platforms/__init__.py:11), hardware libraries inside the detectors (pynvml, amdsmi, libtpu, torch), and envs.VLLM_TARGET_DEVICE.
Not in this fileThe Platform base class and UnspecifiedPlatform (interface.py, page platforms/interface.py: the Platform base class), the vendor classes, and the plugin loader (vllm/plugins/__init__.py, page plugins/__init__.py: the plugin loaders).

The page Platform plugins and current_platform explains the detection from the reader's side, with a worked example. This page reads the file from top to bottom.

Where it sits#

Figure: the detection flow in this file. All line numbers refer to vllm/platforms/__init__.py.
Open full size ↗

The file has three layers. At the bottom are the five detectors (lines 33-228). In the middle, resolve_current_platform_cls_qualname runs them and the out-of-tree detectors, and picks one name (lines 240-293). At the top, __getattr__ turns the name into the one object, once per process (lines 303-326).

Who calls it#

What it calls#

Key classes and functions#

NameLineWhat it does
vllm_version_matches_substr(substr)vllm/platforms/__init__.py:18-30True if the installed vLLM version string contains substr; raises PackageNotFoundError after a warning if vLLM has no package metadata.
tpu_platform_plugin()vllm/platforms/__init__.py:33-54Pathways → the tpu_inference class; import libtpu works → vllm.platforms.tpu.TpuPlatform; else None.
cuda_platform_plugin()vllm/platforms/__init__.py:57-105NVML counts GPUs and the build is not a CPU build → CudaPlatform; Jetson fallback when NVML fails.
rocm_platform_plugin()vllm/platforms/__init__.py:108-140amdsmi finds processors → RocmPlatform; under WSL also a PyTorch HIP check.
xpu_platform_plugin()vllm/platforms/__init__.py:143-162torch.xpu.is_available() → XPUPlatform; sets XPUPlatform.dist_backend to xccl when XCCL exists.
_is_amd_zen_cpu()vllm/platforms/__init__.py:165-171True when /proc/cpuinfo has both AuthenticAMD and avx512.
cpu_platform_plugin()vllm/platforms/__init__.py:174-228VLLM_TARGET_DEVICE=cpu, a CPU build or macOS → CpuPlatform, or ZenCpuPlatform with zentorch.
builtin_platform_pluginsvllm/platforms/__init__.py:231-237Dict from name to detector, in loop order: tpu, cuda, rocm, xpu, cpu.
resolve_current_platform_cls_qualname()vllm/platforms/__init__.py:240-293The detection: CPU short-circuit, run all detectors, pick one name.
_current_platform, _init_tracevllm/platforms/__init__.py:296-297The one object of this process (or None), and the stack of the first read.
__getattr__(name)vllm/platforms/__init__.py:303-326PEP 562 hook: makes current_platform on first read; returns other globals; raises AttributeError for unknown names.
__setattr__(name, value)vllm/platforms/__init__.py:329-336A function with this name; Python does not call a module-level __setattr__ (see the deep dive).
__all__vllm/platforms/__init__.py:339-346The public names, including current_platform.

Walk through the main methods#

cuda_platform_plugin: a detector with error rules#

vllm/platforms/__init__.py:57-105GitHub ↗
def cuda_platform_plugin() -> str | None:
    is_cuda = False
    logger.debug("Checking if CUDA platform is available.")
    try:
        from vllm.utils.import_utils import import_pynvml

        pynvml = import_pynvml()
        pynvml.nvmlInit()
        try:
            # NOTE: Edge case: vllm cpu build on a GPU machine.
            # Third-party pynvml can be imported in cpu build,
            # we need to check if vllm is built with cpu too.
            # Otherwise, vllm will always activate cuda plugin
            # on a GPU machine, even if in a cpu build.
            is_cuda = (
                pynvml.nvmlDeviceGetCount() > 0
                and not vllm_version_matches_substr("cpu")
            )
            if pynvml.nvmlDeviceGetCount() <= 0:
                logger.debug("CUDA platform is not available because no GPU is found.")
            if vllm_version_matches_substr("cpu"):
                logger.debug(
                    "CUDA platform is not available because vLLM is built with CPU."
                )
            if is_cuda:
                logger.debug("Confirmed CUDA platform is available.")
        finally:
            pynvml.nvmlShutdown()
    except Exception as e:
        logger.debug("Exception happens when checking CUDA platform: %s", str(e))
        if "nvml" not in e.__class__.__name__.lower():
            # If the error is not related to NVML, re-raise it.
            raise e

        # CUDA is supported on Jetson, but NVML may not be.
        import os

        def cuda_is_jetson() -> bool:
            return os.path.isfile("/etc/nv_tegra_release") or os.path.exists(
                "/sys/class/tegra-firmware"
            )

        if cuda_is_jetson():
            logger.debug("Confirmed CUDA platform is available on Jetson.")
            is_cuda = True
        else:
            logger.debug("CUDA platform is not available because: %s", str(e))

    return "vllm.platforms.cuda.CudaPlatform" if is_cuda else None
  1. Lines 63-64. Import pynvml and start NVML. This does not start a CUDA context.
  2. Lines 71-74. CUDA is active when NVML counts at least one GPU and the vLLM version does not contain "cpu". The comment (lines 66-70) gives the reason: a CPU build of vLLM on a GPU host must not pick CUDA.
  3. Line 84. nvmlShutdown runs in a finally block.
  4. Lines 87-89. An exception whose class name does not contain "nvml" is raised again. The caller catches it and logs it at DEBUG level (vllm/platforms/__init__.py:260-265).
  5. Lines 94-101. An NVML error on a Jetson (one of two Tegra files exists) still gives CUDA.
  6. Line 105. The result is a dotted class name or None. vllm.platforms.cuda.CudaPlatform is itself an alias that cuda.py sets to the NVML or the non-NVML class (vllm/platforms/cuda.py:1040-1054).

cpu_platform_plugin: three ways to say CPU#

vllm/platforms/__init__.py:174-204GitHub ↗
def cpu_platform_plugin() -> str | None:
    logger.debug("Checking if CPU platform is available.")
    is_cpu = envs.VLLM_TARGET_DEVICE == "cpu"
    if is_cpu:
        logger.debug(
            "Confirmed CPU platform is available because "
            "VLLM_TARGET_DEVICE is set to CPU."
        )
    else:
        try:
            is_cpu = vllm_version_matches_substr("cpu")
            if is_cpu:
                logger.debug(
                    "Confirmed CPU platform is available because vLLM is built "
                    "with CPU."
                )
            if not is_cpu:
                import sys

                is_cpu = sys.platform.startswith("darwin")
                if is_cpu:
                    logger.debug(
                        "Confirmed CPU platform is available because the machine "
                        "is MacOS."
                    )
        except Exception as e:
            logger.debug("CPU platform is not available because: %s", str(e))

    if not is_cpu:
        return None
  1. Line 176. VLLM_TARGET_DEVICE == "cpu" is enough.
  2. Line 184. Else a vLLM version string with "cpu" (a CPU build).
  3. Line 193. Else macOS (sys.platform starts with darwin).
  4. Lines 202-203. None of these: None.

The rest of the function (lines 205-228) returns ZenCpuPlatform on an AMD CPU with AVX-512 when zentorch imports. An ImportError falls back to CpuPlatform with a DEBUG log; an OSError (a build that does not match torch) falls back with a WARNING (vllm/platforms/__init__.py:205-228).

resolve_current_platform_cls_qualname: the decision#

vllm/platforms/__init__.py:250-293GitHub ↗
    platform_plugins = load_plugins_by_group(PLATFORM_PLUGINS_GROUP)

    activated_plugins = []

    for name, func in chain(builtin_platform_plugins.items(), platform_plugins.items()):
        try:
            assert callable(func)
            platform_cls_qualname = func()
            if platform_cls_qualname is not None:
                activated_plugins.append(name)
        except Exception:
            logger.debug(
                "Platform plugin %s failed during detection.",
                name,
                exc_info=True,
            )

    activated_builtin_plugins = list(
        set(activated_plugins) & set(builtin_platform_plugins.keys())
    )
    activated_oot_plugins = list(set(activated_plugins) & set(platform_plugins.keys()))

    if len(activated_oot_plugins) >= 2:
        raise RuntimeError(
            "Only one platform plugin can be activated, but got: "
            f"{activated_oot_plugins}"
        )
    elif len(activated_oot_plugins) == 1:
        platform_cls_qualname = platform_plugins[activated_oot_plugins[0]]()
        logger.info("Platform plugin %s is activated", activated_oot_plugins[0])
    elif len(activated_builtin_plugins) >= 2:
        raise RuntimeError(
            "Only one platform plugin can be activated, but got: "
            f"{activated_builtin_plugins}"
        )
    elif len(activated_builtin_plugins) == 1:
        platform_cls_qualname = builtin_platform_plugins[activated_builtin_plugins[0]]()
        logger.debug(
            "Automatically detected platform %s.", activated_builtin_plugins[0]
        )
    else:
        platform_cls_qualname = "vllm.platforms.interface.UnspecifiedPlatform"
        logger.debug("No platform detected, vLLM is running on UnspecifiedPlatform")
    return platform_cls_qualname
  1. Line 250. The out-of-tree detectors, as a dict from entry-point name to function. Lines 240-248 before it return early for VLLM_TARGET_DEVICE=cpu.
  2. Lines 254-259. chain runs the built-in dict first, then the plugin dict. A name is active when its function returns a value that is not None.
  3. Lines 267-270. Set intersections split the active names. A plugin with the same name as a built-in one (for example cuda) would count in both sets. (Derived from the set logic; no test covers it.)
  4. Lines 272-279. Out-of-tree first: two or more raise; one is called again and wins.
  5. Lines 280-289. Then built-in: two or more raise; one is called again and wins.
  6. Line 291. The fallback name of UnspecifiedPlatform.

__getattr__: the lazy object#

vllm/platforms/__init__.py:303-326GitHub ↗
def __getattr__(name: str):
    if name == "current_platform":
        # lazy init current_platform.
        # 1. out-of-tree platform plugins need `from vllm.platforms import
        #    Platform` so that they can inherit `Platform` class. Therefore,
        #    we cannot resolve `current_platform` during the import of
        #    `vllm.platforms`.
        # 2. when users use out-of-tree platform plugins, they might run
        #    `import vllm`, some vllm internal code might access
        #    `current_platform` during the import, and we need to make sure
        #    `current_platform` is only resolved after the plugins are loaded
        #    (we have tests for this, if any developer violate this, they will
        #    see the test failures).
        global _current_platform
        if _current_platform is None:
            platform_cls_qualname = resolve_current_platform_cls_qualname()
            _current_platform = resolve_obj_by_qualname(platform_cls_qualname)()
            global _init_trace
            _init_trace = "".join(traceback.format_stack())
        return _current_platform
    elif name in globals():
        return globals()[name]
    else:
        raise AttributeError(f"No attribute named '{name}' exists in {__name__}.")
  1. Line 304. Only the name current_platform gets the special path.
  2. Lines 317-321. First read: detect, import the class, call it with no arguments, keep the object, and keep the stack in _init_trace. Later reads return the same object.
  3. Lines 323-324. Other names that exist in the module globals are returned as they are.
  4. Line 326. Unknown names raise AttributeError, as a normal module does.

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

Deep dive

Data flow: types in and out

ValueTypeWhere
a detector resultstr | None: a dotted class name such as "vllm.platforms.rocm.RocmPlatform"vllm/platforms/__init__.py:140
platform_pluginsdict[str, Callable], entry-point name to detectorvllm/platforms/__init__.py:250
activated_pluginslist[str] of active namesvllm/platforms/__init__.py:252
return of resolve_current_platform_cls_qualnamestr, never Nonevllm/platforms/__init__.py:240, vllm/platforms/__init__.py:293
_current_platforman instance of a Platform subclass, or None before the first readvllm/platforms/__init__.py:296, vllm/platforms/__init__.py:319
_init_tracestr: the formatted stack of the first readvllm/platforms/__init__.py:297, vllm/platforms/__init__.py:321

Note the name format: a detector returns module.Class with dots only, because resolve_obj_by_qualname splits at the last dot (vllm/utils/import_utils.py:121). The "module:Class" form of register_model does not work here.

Deep dive

Processes, threads and the module __setattr__

The file has no lock. __getattr__ checks _current_platform is None and then assigns it (lines 317-319), so two threads that read the name for the first time at the same moment could both run the detection. In vllm serve the first read happens during imports, before vLLM starts its threads. (Derived; the code has no comment about threads.)

Each process has its own _current_platform. With the spawn start method (vllm/entrypoints/common/startup.py:147-149), each child starts with None and detects again. With fork, a child would get a copy of the parent's object, if the parent had made it. (Derived from the semantics of fork.)

The function named __setattr__. PEP 562 lets a module define __getattr__ and __dir__. It does not add __setattr__, so Python does not call the function at lines 329-336 when code assigns vllm.platforms.current_platform = x. The assignment makes a normal module attribute, and later reads find it without __getattr__. _current_platform keeps its old value. (Derived from Python semantics; no comment in the code says why the function exists.)

vLLM makes such an assignment once. For vllm bench, the CLI replaces an UnspecifiedPlatform with a CpuPlatform object (vllm/entrypoints/cli/main.py:72-86). The rule above applies to it: the function at lines 329-336 does not run.

Deep dive

A worked example: a CPU build on a GPU host

A CI job installs a CPU wheel of vLLM, whose version is for example "0.x.y+cpu", on a host with 4 NVIDIA GPUs and pynvml installed. VLLM_TARGET_DEVICE is not set, so its value is "cuda" (vllm/envs.py:622). (The version string is an example.)

  1. No short-circuit: the value is not cpu (line 244).
  2. tpu: import libtpu fails → None, DEBUG log.
  3. cuda: NVML counts 4 GPUs, but the version contains "cpu", so is_cuda is False → None (lines 71-74, 77-80).
  4. rocm, xpu: None.
  5. cpu: the version contains "cpu" → "vllm.platforms.cpu.CpuPlatform" (lines 184-189).
  6. One built-in name is active → cpu_platform_plugin() runs again, DEBUG log "Automatically detected platform cpu." (lines 285-289).

The comment at lines 241-243 describes the case that the short-circuit is for: CPU-only jobs that reuse an accelerator wheel. There, VLLM_TARGET_DEVICE=cpu makes the CPU detector answer (line 176). The CUDA detector answers too, because the version has no "cpu". Without the short-circuit, lines 280-284 would raise RuntimeError. (Derived.)

Deep dive

Error paths: what raises, what only logs

SituationResultWhere
A detector raisesDEBUG log with traceback; not activevllm/platforms/__init__.py:260-265
A plugin object is not callablethe assert fails inside try: DEBUG log; not activevllm/platforms/__init__.py:255-256
Two or more out-of-tree detectors activeRuntimeErrorvllm/platforms/__init__.py:272-276
No out-of-tree, two or more built-in activeRuntimeErrorvllm/platforms/__init__.py:280-284
The winner raises on its second callnot caught; the read of current_platform failsvllm/platforms/__init__.py:278, vllm/platforms/__init__.py:286
No detector activeUnspecifiedPlatform, DEBUG logvllm/platforms/__init__.py:290-292
vLLM has no package metadataWARNING, then PackageNotFoundError. The CUDA detector raises it again and the loop catches it; the CPU detector catches it itself (DEBUG log)vllm/platforms/__init__.py:22-29, vllm/platforms/__init__.py:87-89, vllm/platforms/__init__.py:199-200
zentorch fails with OSErrorWARNING; CpuPlatformvllm/platforms/__init__.py:218-226
An unknown attribute of the moduleAttributeErrorvllm/platforms/__init__.py:325-326
Deep dive

Environment variables

VariableDefaultRead atEffect
VLLM_TARGET_DEVICE"cuda", lower-cased (vllm/envs.py:622)vllm/platforms/__init__.py:176, vllm/platforms/__init__.py:244cpu: CPU platform with no detection loop.
VLLM_TPU_USING_PATHWAYStrue if JAX_PLATFORMS contains "proxy" (vllm/envs.py:1594-1596)vllm/platforms/__init__.py:37The TPU detector returns the tpu_inference class at once.
VLLM_PLUGINSnot setthrough load_plugins_by_group (vllm/plugins/__init__.py:40)Filters the out-of-tree detectors.

Check yourself#

  1. 1
    Which function returns the dotted class name of the platform, and which code makes the object?
    easy
    Show answer

    resolve_current_platform_cls_qualname() returns the name (vllm/platforms/__init__.py:240-293). The module __getattr__ imports the class and calls it once per process (vllm/platforms/__init__.py:318-319).

  2. 2
    A CPU build of vLLM runs on a GPU host. Why does the CUDA detector not answer?
    medium
    Show answer

    It answers only when NVML counts GPUs and vllm_version_matches_substr("cpu") is false (vllm/platforms/__init__.py:71-74). A CPU build has "cpu" in its version, so the detector returns None, and the CPU detector answers (vllm/platforms/__init__.py:184).

  3. 3
    An out-of-tree plugin object in vllm.platform_plugins is a string, not a function. What happens?
    medium
    Show answer

    assert callable(func) fails inside the try block, so the loop logs it at DEBUG level and treats it as not active (vllm/platforms/__init__.py:255-265). Detection goes on with the other detectors.

  4. 4
    A test runs vllm.platforms.current_platform = MyPlatform(). Does _current_platform change, and what do modules that imported current_platform earlier see?
    hard
    Show answer

    No. Python does not call a module-level __setattr__ (PEP 562 covers only __getattr__ and __dir__), so the function at vllm/platforms/__init__.py:329-336 does not run. The assignment makes a normal module attribute; new reads of vllm.platforms.current_platform get MyPlatform(). Modules that ran from vllm.platforms import current_platform before keep their own reference to the old object. (Derived from Python semantics; see the deep dive.)

  • Platform plugins and current_platformEach vLLM process makes one Platform object, current_platform, the first time code reads it. Five built-in detectors and the out-of-tree detectors from vllm.platform_plugins decide its class; one out-of-tree platform wins over the built-in ones.
  • The Platform interfaceThe Platform class is the contract between vLLM and a hardware vendor. Identity attributes describe the device; config hooks change VllmConfig in the front end; component pickers name the classes to use; kernel hooks load kernels. CudaPlatform is the reference implementation.
  • How vLLM finds and loads pluginsvLLM finds plugins as Python entry points in six groups. Each process of vllm serve loads only the groups that it needs, at its own time. One environment variable, VLLM_PLUGINS, filters five of them.
  • All the ways to extend vLLMvLLM has about twenty extension points of three kinds - six entry-point groups, a dozen registries that a general plugin fills, and config fields that name your class. This page maps each one to its process and its page.
  • platforms/interface.py: the Platform base classvllm/platforms/interface.py defines Platform, the base class of every hardware platform, with 12 class attributes and 90 methods, plus PlatformEnum, CpuArchEnum, DeviceCapability, the device-id helpers and UnspecifiedPlatform.
  • plugins/__init__.py: the plugin loadersvllm/plugins/__init__.py names the five plugin entry-point groups and has the three loaders: the shared group loader, the once-per-process general loader and the strict endpoint loader.