plugins/__init__.py: the plugin loaders

Code version: vLLM @ 3e44c0d · 2026-10-22

Purpose#

vllm/plugins/__init__.py finds the plugins that other Python packages publish as entry points, filters them with VLLM_PLUGINS, and imports them for the code that asks.

The file only finds, filters and imports. Each caller decides what to do with the loaded objects. The page How vLLM finds and loads plugins explains the same code from the view of the processes. This page reads the file line by line.

Where it sits#

Read panel 1 row by row. Green is the path of the general plugins, blue is the shared loader, and pink is the path of the endpoint plugins. Grey boxes are code outside the plugin loaders. Panel 2 is a short form of the walk-through below.

Who calls it#

load_general_plugins() has five callers in vllm/:

• AsyncEngineArgs.add_cli_args in the front end, before the CLI parser gets its flags. The comment says that a plugin can add a choice to --quantization or --device (vllm/engine/arg_utils.py:3240-3243).
• EngineArgs.__post_init__, again in the front end, and also for the offline LLM class, which builds no CLI parser (vllm/engine/arg_utils.py:921-924). The same file also imports the function at module level (vllm/engine/arg_utils.py:123).
• EngineCore.__init__ in the engine core process, before it makes the executor (vllm/v1/engine/core.py:129-132, vllm/v1/engine/core.py:147).
• WorkerWrapperBase.init_worker in each worker, before it resolves worker_cls (vllm/v1/worker/worker_base.py:282-284, vllm/v1/worker/worker_base.py:292-296).
• _run() of the model registry subprocess, before it runs the function that the parent process sent (vllm/model_executor/models/registry.py:1608-1616). vLLM starts this subprocess only when it cannot use the cached model information: the cache has no valid entry, or vLLM cannot find the source file of the model (vllm/model_executor/models/registry.py:1086-1109). This is true for built-in models too, because every built-in model is a lazy registry entry (vllm/model_executor/models/registry.py:1565-1573).

load_plugins_by_group() has three callers outside this file. Each one imports the group constant from this file:

• Platform detection. resolve_current_platform_cls_qualname() loads vllm.platform_plugins (vllm/platforms/__init__.py:10, vllm/platforms/__init__.py:250). It runs on the first read of current_platform in a process (vllm/platforms/__init__.py:316-319). With VLLM_TARGET_DEVICE=cpu, the function returns the CPU platform before it loads any platform plugin (vllm/platforms/__init__.py:244-248).
• Stat loggers. load_stat_logger_plugin_factories() loads vllm.stat_logger_plugins (vllm/v1/metrics/loggers.py:84-98). AsyncLLM.__init__ calls it (vllm/v1/engine/async_llm.py:155-156).
• IO processors. get_io_processor() loads vllm.io_processor_plugins, but only when a plugin name comes from the model config (io_processor_plugin) or from the Hugging Face config of the model (vllm/plugins/io_processors/__init__.py:42-61). The pooling serving code calls it (vllm/entrypoints/pooling/pooling/io_processor.py:51-55).

load_endpoint_plugins() has one caller: attach_endpoint_plugins() (vllm/plugins/endpoint_plugins/interface.py:106-108). build_app calls that function after the core routes (vllm/entrypoints/launchers/app.py:45-51).

Tests also call the loaders. For example, a test resets the flag plugins_loaded by hand, so that load_general_plugins() runs again (tests/v1/kv_connector/unit/test_scheduler_kv_connector_override.py:113-119).

What it calls#

• importlib.metadata.entry_points(group=...), imported inside the functions (vllm/plugins/__init__.py:38, vllm/plugins/__init__.py:120).
• vllm.envs for VLLM_PLUGINS (vllm/plugins/__init__.py:8, vllm/plugins/__init__.py:40, vllm/plugins/__init__.py:122).
• EntryPoint.load(), which imports the module of the plugin (vllm/plugins/__init__.py:69).
• The plugin objects: each general plugin function (vllm/plugins/__init__.py:89-90) and each endpoint plugin factory (vllm/plugins/__init__.py:138).
• The standard logging module. The file uses logging.getLogger(__name__) (vllm/plugins/__init__.py:14), not vLLM's init_logger, which adds helper methods to the same kind of logger (vllm/logger.py:328-335).

The subfolders of vllm/plugins/#

Three subfolders hold plugin code that vLLM itself ships. The page Plugins in the API server explains how to write such plugins.

• lora_resolvers/: two general plugins that vLLM publishes in its own pyproject.toml (pyproject.toml:45-47). Each one registers a LoRA resolver. register_filesystem_resolver registers a resolver only when VLLM_LORA_RESOLVER_CACHE_DIR is set (vllm/plugins/lora_resolvers/filesystem_resolver.py:49-61). The resolver accepts a folder when its adapter_config.json has peft_type LORA and the same base model (vllm/plugins/lora_resolvers/filesystem_resolver.py:30-45). register_hf_hub_resolver registers a resolver for Hugging Face repositories only when VLLM_LORA_RESOLVER_HF_REPO_LIST is set and VLLM_PLUGINS names lora_hf_hub_resolver (vllm/plugins/lora_resolvers/hf_hub_resolver.py:131-145). OpenAIServingModels in the API server gets the registered resolvers (vllm/entrypoints/openai/models/serving.py:113-117).
• io_processors/: the abstract class IOProcessor (vllm/plugins/io_processors/interface.py:18-24), with the abstract methods pre_process and post_process (vllm/plugins/io_processors/interface.py:41-65), and the loader get_io_processor(). An IO processor plugin changes the input and the output of a pooling model. The model config can name the plugin in io_processor_plugin (vllm/config/model.py:410).
• endpoint_plugins/: the protocol EndpointPlugin (vllm/plugins/endpoint_plugins/interface.py:49-94) and the two wiring phases. Phase A, attach_endpoint_plugins(), adds the routes of each endpoint plugin to the FastAPI app (vllm/plugins/endpoint_plugins/interface.py:97-146). Phase B, init_endpoint_plugins_state(), calls init_state of each plugin (vllm/plugins/endpoint_plugins/interface.py:149-164). init_app_state runs Phase B (vllm/entrypoints/launchers/api_server/app_state.py:173). The render server runs it too, with no engine client (vllm/entrypoints/launchers/render/app_state.py:124).

Key classes and functions#

Walk through the main methods#

load_plugins_by_group: find, filter, import#

def load_plugins_by_group(group: str) -> dict[str, Callable[[], Any]]:
    """Load plugins registered under the given entry point group."""
    from importlib.metadata import entry_points

    allowed_plugins = envs.VLLM_PLUGINS

    discovered_plugins = entry_points(group=group)
    if len(discovered_plugins) == 0:
        logger.debug("No plugins for group %s found.", group)
        return {}

    # Check if the only discovered plugin is the default one
    is_default_group = group == DEFAULT_PLUGINS_GROUP
    # Use INFO for non-default groups and DEBUG for the default group
    log_level = logger.debug if is_default_group else logger.info

    log_level("Available plugins for group %s:", group)
    for plugin in discovered_plugins:
        log_level("- %s -> %s", plugin.name, plugin.value)

    if allowed_plugins is None:
        log_level(
            "All plugins in this group will be loaded. "
            "Set `VLLM_PLUGINS` to control which plugins to load."
        )

    plugins = dict[str, Callable[[], Any]]()
    for plugin in discovered_plugins:
        if allowed_plugins is None or plugin.name in allowed_plugins:
            if allowed_plugins is not None:
                log_level("Loading plugin %s", plugin.name)

            try:
                func = plugin.load()
                plugins[plugin.name] = func
            except Exception:
                logger.exception("Failed to load plugin %s", plugin.name)

    return plugins

1. Line 38. The import of entry_points is inside the function, so it runs only on a call.
2. Line 40. envs.VLLM_PLUGINS is None when the variable is not set, else the value split at commas (vllm/envs.py:1163-1167).
3. Lines 42-45. entry_points(group=group) returns the entry points of all installed packages in the group. No entry points: a DEBUG log and an empty dict.
4. Lines 48-50. The log level is DEBUG for vllm.general_plugins and INFO for every other group. The comment on line 47 does not match the code: the check is only for the group name.
5. Lines 52-60. The function logs each found plugin as name -> value. When the allowlist is not set, it also logs that it loads all plugins.
6. Lines 63-66. A plugin passes when there is no allowlist, or when its entry-point name is in the list. The test is an exact string match. With an allowlist, line 66 logs "Loading plugin" for each name that passes. A name that does not pass gets no such line.
7. Lines 68-72. plugin.load() imports the module and gets the attribute. An exception is logged with its traceback, and the loop continues.
8. Line 74. The result maps the entry-point name to the loaded object. The function does not call the objects.

load_general_plugins: once per process#

def load_general_plugins():
    """WARNING: plugins can be loaded for multiple times in different
    processes. They should be designed in a way that they can be loaded
    multiple times without causing issues.
    """
    global plugins_loaded
    if plugins_loaded:
        return
    plugins_loaded = True

    plugins = load_plugins_by_group(group=DEFAULT_PLUGINS_GROUP)
    # general plugins, we only need to execute the loaded functions
    for func in plugins.values():
        func()

1. Lines 78-80. The docstring warns that plugins can load many times, in different processes. So a plugin function must be safe to run again.
2. Lines 82-85. The guard. The first call sets the module global; each later call in the same process returns at once.
3. Line 87. The shared loader gets the functions of vllm.general_plugins, filtered by VLLM_PLUGINS.
4. Lines 89-90. Each function runs with no arguments and no try block. An exception goes up to the caller, for example EngineCore.__init__.

load_endpoint_plugins: opt-in only#

The docstring gives the reason for a stricter rule: endpoint plugins add HTTP routes, so a plugin loads only when VLLM_PLUGINS names it (vllm/plugins/__init__.py:98-103).

    from importlib.metadata import entry_points

    if envs.VLLM_PLUGINS is None:
        discovered = entry_points(group=ENDPOINT_PLUGINS_GROUP)
        if discovered:
            logger.warning(
                "Found endpoint plugin(s) %s but VLLM_PLUGINS is not set. "
                "Endpoint plugins add HTTP routes and must be explicitly "
                "allowlisted via VLLM_PLUGINS to be loaded.",
                [p.name for p in discovered],
            )
        return []

    factories = load_plugins_by_group(ENDPOINT_PLUGINS_GROUP)

    endpoint_plugins: list[EndpointPlugin] = []
    for name, factory in factories.items():
        try:
            plugin = factory()
        except Exception:
            logger.exception("Failed to instantiate endpoint plugin %s", name)
            continue

        required_tasks = plugin.required_tasks
        if required_tasks is not None and (
            supported_tasks is None or not set(required_tasks) & set(supported_tasks)
        ):
            logger.info(
                "Skipping endpoint plugin %s: requires one of tasks %s, "
                "server supports %s",
                name,
                required_tasks,
                supported_tasks,
            )
            continue

        logger.info("Loaded endpoint plugin %s", name)
        endpoint_plugins.append(plugin)

    return endpoint_plugins

1. Lines 122-131. VLLM_PLUGINS not set: the function still lists the group. If it finds plugins, it logs a WARNING with their names. Then it returns an empty list. So an endpoint plugin never loads by accident.
2. Line 133. VLLM_PLUGINS set: the shared loader imports the named entry points. An empty string gives [""], which matches no name (vllm/plugins/__init__.py:105-109).
3. Lines 136-141. Each factory runs with no arguments. If it raises, the function logs the error and skips the plugin.
4. Lines 143-154. The task gate. required_tasks = None means "always eligible". Otherwise, the plugin needs at least one task in common with supported_tasks. If the caller gives supported_tasks = None, every plugin with tasks is skipped. A skip writes an INFO log.
5. Lines 156-159. The function logs each loaded plugin and returns them in the order of discovery.

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

Related concepts#

• How vLLM finds and loads plugins: the same code from the view of the processes, with fork and spawn.
• All the ways to extend vLLM: all extension points, with and without entry points.
• Platform plugins and current_platform: what the platform code does with the loaded detectors.
• Plugins in the API server: endpoint plugins, IO processors, stat loggers and LoRA resolvers.
• Serving LoRA adapters: from the model name to the wrapped layer: how the API server uses the LoRA resolvers.
• Python entry points: the Python packaging basics under all of this.
• Tour: Life of a plugin and Tour: Build a toy plugin: the same code as guided tours.