The Platform interface

Code version: vLLM @ 3e44c0d · 2026-10-22

The idea#

The page Platform plugins and current_platform explains how a process chooses its platform. This page explains what the chosen object must do. Platform in vllm/platforms/interface.py is a class with 90 methods (84 without a leading underscore) and 12 class attributes. Most have a default. A vendor subclass overrides the ones that differ on its hardware.

The members fall into four groups:

1. Identity attributes. Values that describe the device: its name, its PyTorch device type, its dispatch key, its distributed backend, and more.
2. Config hooks. A platform hook that vLLM calls while it builds VllmConfig. The platform can check the config, change defaults, or reject a setting. check_and_update_config is the most important one: it sets the worker class.
3. Component pickers. A component picker returns the dotted name of a class: the attention backend, the device communicator, the LoRA kernel wrapper, the graph wrapper. vLLM calls it where it builds that component.
4. Kernel hooks. Methods that import kernel libraries or replace Triton kernels.

Anything that the class does not define goes to a fallback: Platform.__getattr__ looks for the name in torch.<device_type>, for example torch.cuda.

An everyday comparison. Think of a franchise manual. Head office writes the chapters and a default answer for each one. A local shop rewrites only the chapters that differ in its country. Head office reads each chapter at a fixed moment: the opening checklist before the shop opens, the staff chapter when it hires. The comparison stops here. In vLLM each process reads its own copy of the manual. The front end reads the config chapters; a worker reads the chapters about the model and the GPU.

The picture#

Read the numbers in order. Steps 1-6 run once, in the front end, and shape the VllmConfig. The engine core and the workers get that config through pickling, and they detect the platform again for themselves. The pickers in the worker column run in each worker.

In the code#

Step 1: identity attributes#

class Platform:
    _enum: PlatformEnum
    device_name: str
    device_type: str

    # available dispatch keys:
    # check https://github.com/pytorch/pytorch/blob/313dac6c1ca0fa0cde32477509cce32089f8532a/torchgen/model.py#L134 # noqa
    # use "CPU" as a fallback for platforms not registered in PyTorch
    dispatch_key: str = "CPU"

    # available ray device keys:
    # https://github.com/ray-project/ray/blob/10ba5adadcc49c60af2c358a33bb943fb491a171/python/ray/_private/ray_constants.py#L438 # noqa
    # empty string means the device does not support ray
    ray_device_key: str = ""

    @classmethod
    def log_warnings(cls) -> None:
        """Log platform-specific diagnostics after logging is configured."""

    # platform-agnostic way to specify the device control environment variable,
    # .e.g. CUDA_VISIBLE_DEVICES for CUDA.
    # hint: search for "get_visible_accelerator_ids_env_var" in
    # https://github.com/ray-project/ray/tree/master/python/ray/_private/accelerators # noqa
    device_control_env_var: str = "VLLM_DEVICE_CONTROL_ENV_VAR_PLACEHOLDER"

    # environment variables that need to be set to 1 to prevent ray from
    # setting the visible devices e.g.
    # RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES
    ray_noset_device_env_vars: list[str] = []

    # The torch.compile backend for compiling simple and
    # standalone functions. The default value is "inductor" to keep
    # the same behavior as PyTorch.
    # NOTE: for the forward part of the model, vLLM has another separate
    # compilation strategy.
    simple_compile_backend: str = "inductor"

    # The backend used for distributed communication.
    dist_backend: str = ""

    supported_quantization: list[str] = []

    additional_env_vars: list[str] = []

    _global_graph_pool: Any | None = None

The table compares the defaults with CUDA (vllm/platforms/cuda.py:226-236):

Step 2: config hooks, and when each one runs#

Four hooks change the config. Three of them run in the front end, while vllm serve parses the flags and builds VllmConfig:

1. pre_register_and_update(parser) runs in AsyncEngineArgs.add_cli_args, after the general plugins and after the parser has its flags (vllm/engine/arg_utils.py:3243-3257). The docstring names the use: register an out-of-tree quantization config (vllm/platforms/interface.py:638-651). It runs a second time, with no parser, at the start of create_engine_config (vllm/engine/arg_utils.py:2246).
2. apply_config_platform_defaults(vllm_config) runs in VllmConfig.__post_init__ (vllm/config/vllm.py:1804). It changes defaults, for example the list of custom ops. ROCm uses it to turn on custom ops for its AITER kernels (vllm/platforms/rocm.py:1074-1084).
3. check_and_update_config(vllm_config) runs late in VllmConfig.__post_init__ (vllm/config/vllm.py:2046). The method goes on after it, to line 2318 (vllm/config/vllm.py:2318). It can raise for a setting that the hardware does not support, or change the config. Its most important job is worker_cls. CUDA shows the pattern:

    @classmethod
    def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
        parallel_config = vllm_config.parallel_config
        model_config = vllm_config.model_config

        if parallel_config.worker_cls == "auto":
            parallel_config.worker_cls = "vllm.v1.worker.gpu_worker.Worker"

        scheduler_config = vllm_config.scheduler_config
        # Note: model_config may be None during testing
        if (
            model_config is not None
            and model_config.is_mm_prefix_lm
            and scheduler_config.is_multimodal_model
            and not scheduler_config.disable_chunked_mm_input
        ):
            logger.warning_once(
                "Forcing --disable_chunked_mm_input for models "
                "with multimodal-bidirectional attention."
            )
            scheduler_config.disable_chunked_mm_input = True

The fourth config hook, update_block_size_for_backend, runs later, after the model is loaded, because it needs the attention backends of the layers (vllm/platforms/interface.py:749-794). With worker processes, each worker calls it (vllm/v1/executor/multiproc_executor.py:701-702). With the uni executor, the engine core process calls it (vllm/v1/executor/uniproc_executor.py:74-75). When the user did not set --block-size, it picks a block size that all attention backends accept. That is the preferred size of a single backend, else the default 16 if all accept it, else the smallest common size (vllm/platforms/interface.py:771-781, vllm/platforms/interface.py:721-739).

The front end also calls validate_environ, which warns about unknown VLLM_* variables (vllm/engine/arg_utils.py:2250, vllm/platforms/interface.py:1454-1468).

Step 3: component pickers#

A picker returns a dotted class name, and the caller imports it with resolve_obj_by_qualname. The attention selector is the main example:

@cache
def _cached_get_attn_backend(
    backend,
    attn_selector_config: AttentionSelectorConfig,
    num_heads: int | None = None,
    *,
    _run_kv_cache_dtype: CacheDType | None = None,
) -> type[AttentionBackend]:
    # Some platform selectors inspect the run-wide cache dtype. Keep it in the
    # cache key even though they read the value from the current config.
    from vllm.platforms import current_platform

    attention_cls = current_platform.get_attn_backend_cls(
        backend,
        attn_selector_config=attn_selector_config,
        num_heads=num_heads,
    )
    if not attention_cls:
        raise ValueError(
            f"Invalid attention backend for {current_platform.device_name}"
        )
    backend = resolve_obj_by_qualname(attention_cls)
    return backend

The table lists every picker that this page covers, with its caller.

opaque_attention_op is a yes/no question, not a class name, but it picks a path too. True means that vLLM registers attention as one opaque custom op for torch.compile. False means that the layer calls the backend directly, and torch.compile traces it (vllm/model_executor/layers/attention/attention.py:436-440).

Step 4: kernel hooks#

• import_kernels imports the compiled kernel libraries. The default imports vllm._C and, if present, vllm._moe_C_stable_libtorch (vllm/platforms/interface.py:365-373). It runs when Python imports vllm._custom_ops, at module level (vllm/_custom_ops.py:21). CUDA imports its own set (vllm/platforms/cuda.py:238-250).
• register_triton_kernel_overrides lets a platform replace core Triton kernels. The worker calls it right after the general plugins, before it makes the worker class (vllm/v1/worker/worker_base.py:286-290). The default does nothing (vllm/platforms/interface.py:375-386). The CPU platform registers C++ versions, but only when Triton is missing (vllm/platforms/cpu.py:690-705).
• import_ir_kernels imports vllm.kernels, the IR op implementations. The docstring says that an out-of-tree platform should override it (vllm/platforms/interface.py:258-264). The kernel config calls it (vllm/config/kernel.py:54).

Step 5: the fallback to torch.<device_type>#

    def __getattr__(self, key: str):
        # Pickle checks dunder methods like __getstate__. If we return None
        # for them, pickle treats it like a real value and tries to call it.
        if key.startswith("__") and key.endswith("__"):
            raise AttributeError(key)

        device = getattr(torch, self.device_type, None)
        if device is not None and hasattr(device, key):
            attr = getattr(device, key)
            # NOTE: `hasattr(device, key)=True` can only avoid AttributeError,
            # but the value of this attr could be `None`.
            if attr is not None:
                return attr

        logger.warning_once(
            "Current platform %s does not have '%s' attribute.",
            self.device_type,
            key,
        )
        return None

1. Lines 1303-1304. Dunder names raise AttributeError at once. The comment says why: pickle looks for names such as __getstate__.
2. Lines 1306-1312. The method looks up torch.cuda (for CUDA) and returns the attribute, if it is not None.
3. Lines 1314-1319. Otherwise it logs a warning once and returns None. It does not raise.

So current_platform.synchronize is torch.cuda.synchronize on CUDA, and the forward context checks for None before the call (vllm/forward_context.py:368-370). CudaPlatform does not define synchronize itself; the fallback finds it. (Derived from the method list of cuda.py.)

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

What can go wrong#

• A missing method returns None. Platform.__getattr__ logs "does not have ... attribute" once and returns None (vllm/platforms/interface.py:1314-1319). The caller then fails later with a TypeError if it calls the value. (Derived.)
• LoRA on a new platform. The base get_punica_wrapper raises NotImplementedError (vllm/platforms/interface.py:1203-1206). Implement it, or do not use --enable-lora.
• A selected backend is invalid. On CUDA, --attention-backend with an invalid backend raises ValueError; vLLM does not fall back (vllm/platforms/cuda.py:430-446).
• The block sizes do not fit together. When the attention backends share no block size, update_block_size_for_backend raises ValueError (vllm/platforms/interface.py:740-747).
• A quantization method is not supported. If the platform sets supported_quantization and the method is not in it, verify_quantization raises ValueError (vllm/platforms/interface.py:1153-1158).

Why it matters. A vendor port of vLLM is mostly a Platform subclass plus a worker class. Knowing which hook runs in which process, and when, tells you where to put each change and where to look when start-up fails.