How vLLM finds and loads plugins

Code version: vLLM @ 3e44c0d · 2026-10-22

The idea#

A plugin is code in another Python package that changes vLLM. For example, a plugin can add a model architecture, a hardware platform or a new HTTP route. vLLM does not import your package by its name. Your package publishes an entry point in its metadata, under a plugin group such as vllm.general_plugins. vLLM asks Python which installed packages publish a name in that group, and then imports the object that the name points to. The page Python entry points explains entry points in general.

vllm serve runs as several processes. The front end runs the HTTP server. The EngineCore process runs the scheduler. Each worker process owns one GPU. With tensor parallelism, one model runs on several GPUs, so there are several workers (Parallelism basics↗).

Python objects live in one process. When a plugin registers a model in the front end, the registry in a worker does not change. So each process must load the plugins itself. But not every process needs every group. Only the front end needs HTTP routes, and only the workers run the model. So vLLM loads each group in the processes that use it, at the moment that it needs it.

An everyday comparison. Think of a company with three offices: sales, planning and the factory. Each office has its own copy of the rule book. When a new rule comes, each office must add it to its own copy. The sales office reads the rules for customers. The factory does not need them. The comparison stops here. In vLLM, each process reads the same central list: the installed entry points. No process sends the rules to the next one.

Three rules control the loading:

1. One allowlist for five groups. The environment variable VLLM_PLUGINS lists the plugin names to load. If it is not set, vLLM loads every plugin that it finds, but not the endpoint plugins. The group vllm.logits_processors does not use this list (Step 7).
2. Once per process. General plugins run at most once in each process. A module-level flag stops the second load.
3. A broken import is skipped. A broken plugin function stops start-up. vLLM logs and skips an entry point that fails to import. It does not catch an error that a general plugin function raises.

The picture#

Read panel 1 column by column. Green groups load in every process. Blue groups load only in the front end. The violet group loads in the front end and in the workers. Panel 2 shows the shared loader. Panel 3 shows that a new Python process always loads the general plugins again, and a second call in the same process does nothing.

In the code#

Step 1: the plugin groups#

Five group names are constants in vllm/plugins/__init__.py. Their comments say where each group loads:

# Default plugins group will be loaded in all processes(process0, engine core
# process and worker processes)
DEFAULT_PLUGINS_GROUP = "vllm.general_plugins"
# IO processor plugins group will be loaded in process0 only
IO_PROCESSOR_PLUGINS_GROUP = "vllm.io_processor_plugins"
# Platform plugins group will be loaded in all processes when
# `vllm.platforms.current_platform` is called and the value not initialized,
PLATFORM_PLUGINS_GROUP = "vllm.platform_plugins"
# Stat logger plugins group will be loaded in process0 only when serve vLLM with
# async mode.
STAT_LOGGER_PLUGINS_GROUP = "vllm.stat_logger_plugins"
# Endpoint plugins group is loaded in the API server front end process only.
# Each entry point resolves to a factory returning an `EndpointPlugin`
# (see `vllm/plugins/endpoint_plugins/interface.py`).
ENDPOINT_PLUGINS_GROUP = "vllm.endpoint_plugins"

# make sure one process only loads plugins once
plugins_loaded = False

A sixth group, vllm.logits_processors, is defined in the sampler code (vllm/v1/sample/logits_processor/__init__.py:48). The Model Runner V2 loader defines the same name again (vllm/v1/worker/gpu/sample/logits_processor/loader.py:31-34). vLLM itself publishes two general plugins, the two LoRA resolvers:

[project.entry-points."vllm.general_plugins"]
lora_filesystem_resolver = "vllm.plugins.lora_resolvers.filesystem_resolver:register_filesystem_resolver"
lora_hf_hub_resolver = "vllm.plugins.lora_resolvers.hf_hub_resolver:register_hf_hub_resolver"

The table shows what the object behind each entry point must be, and who loads the group.

Step 2: one loader for five groups#

Five groups use one function, load_plugins_by_group(). It finds the entry points of one group, filters them by name, and imports them:

def load_plugins_by_group(group: str) -> dict[str, Callable[[], Any]]:
    """Load plugins registered under the given entry point group."""
    from importlib.metadata import entry_points

    allowed_plugins = envs.VLLM_PLUGINS

    discovered_plugins = entry_points(group=group)
    if len(discovered_plugins) == 0:
        logger.debug("No plugins for group %s found.", group)
        return {}

    # Check if the only discovered plugin is the default one
    is_default_group = group == DEFAULT_PLUGINS_GROUP
    # Use INFO for non-default groups and DEBUG for the default group
    log_level = logger.debug if is_default_group else logger.info

    log_level("Available plugins for group %s:", group)
    for plugin in discovered_plugins:
        log_level("- %s -> %s", plugin.name, plugin.value)

    if allowed_plugins is None:
        log_level(
            "All plugins in this group will be loaded. "
            "Set `VLLM_PLUGINS` to control which plugins to load."
        )

    plugins = dict[str, Callable[[], Any]]()
    for plugin in discovered_plugins:
        if allowed_plugins is None or plugin.name in allowed_plugins:
            if allowed_plugins is not None:
                log_level("Loading plugin %s", plugin.name)

            try:
                func = plugin.load()
                plugins[plugin.name] = func
            except Exception:
                logger.exception("Failed to load plugin %s", plugin.name)

    return plugins

The function returns a dict from plugin name to the loaded object (vllm/plugins/__init__.py:62). It does not call the objects. The caller decides what to do: load_general_plugins() calls each function, and the platform code calls each detector function (vllm/platforms/__init__.py:254-257).

Step 3: the allowlist VLLM_PLUGINS#

    # a list of plugin names to load, separated by commas.
    # if this is not set, it means all plugins will be loaded
    # if this is set to an empty string, no plugins will be loaded
    "VLLM_PLUGINS": lambda: (
        None
        if "VLLM_PLUGINS" not in os.environ
        else os.environ["VLLM_PLUGINS"].split(",")
    ),

Line 64 of the loader compares plugin.name with the list. This is the name on the left side of the entry point, for example lora_filesystem_resolver in pyproject.toml. It is not the name of the Python package.

Step 4: general plugins, once per process#

def load_general_plugins():
    """WARNING: plugins can be loaded for multiple times in different
    processes. They should be designed in a way that they can be loaded
    multiple times without causing issues.
    """
    global plugins_loaded
    if plugins_loaded:
        return
    plugins_loaded = True

    plugins = load_plugins_by_group(group=DEFAULT_PLUGINS_GROUP)
    # general plugins, we only need to execute the loaded functions
    for func in plugins.values():
        func()

The module-level flag plugins_loaded makes a second call in the same process return at once (lines 83-84). The function sets the flag before it calls any plugin function (line 85). Then it calls each function, with no try block (lines 89-90). The docstring warns that a plugin can load several times, in different processes (lines 78-80). So a plugin function must be safe to run again. vLLM's own test plugin checks the registry before it registers a model:

def register():
    # Test directly passing the model
    from .my_opt import MyOPTForCausalLM, MyOPTRawInputTokensForCausalLM

    if "MyOPTForCausalLM" not in ModelRegistry.get_supported_archs():
        ModelRegistry.register_model("MyOPTForCausalLM", MyOPTForCausalLM)

Step 5: which process loads what, and when#

Five call sites run load_general_plugins(), in four kinds of process: the front end (two calls), the engine core, the worker and the model registry subprocess.

Front end. The CLI builds the parsers of all subcommands before it reads your flags (vllm/entrypoints/cli/main.py:100-110). The serve parser calls make_arg_parser (vllm/entrypoints/cli/serve.py:172), which calls AsyncEngineArgs.add_cli_args (vllm/entrypoints/common/cli_args.py:466). That function loads the general plugins first:

    def add_cli_args(
        parser: FlexibleArgumentParser, async_args_only: bool = False
    ) -> FlexibleArgumentParser:
        # Initialize plugin to update the parser, for example, The plugin may
        # add a new kind of quantization method to --quantization argument or
        # a new device to --device argument.
        load_general_plugins()

Later, EngineArgs.__post_init__ calls load_general_plugins() again (vllm/engine/arg_utils.py:921-924). In the front end of vllm serve, this second call returns at once because of the flag. It is still useful: the offline LLM class makes an EngineArgs object (vllm/entrypoints/llm.py:295) and does not build a CLI parser.

Engine core process. EngineCore.__init__ loads the general plugins before it makes the executor and the scheduler:

        # plugins need to be loaded at the engine/scheduler level too
        from vllm.plugins import load_general_plugins

        load_general_plugins()

The executor comes later, at vllm/v1/engine/core.py:147. So the plugins are ready before any worker starts.

Worker. Each worker loads them in WorkerWrapperBase.init_worker, before it makes the worker class:

    def init_worker(self, all_kwargs: list[dict[str, Any]]) -> None:
        """Here we inject some common logic before initializing the worker.
        Arguments are passed to the worker class constructor.
        """
        kwargs = all_kwargs[self.rpc_rank]

        vllm_config: VllmConfig | None = kwargs.get("vllm_config")
        assert vllm_config is not None, (
            "vllm_config is required to initialize the worker"
        )
        self.vllm_config = vllm_config

        vllm_config.enable_trace_function_call_for_thread()

        from vllm.plugins import load_general_plugins

        load_general_plugins()

        # Let the platform replace core Triton kernels (e.g. CPU fallback
        # implementations) before any worker or model code can launch them.
        from vllm.platforms import current_platform

        current_platform.register_triton_kernel_overrides()

A worker process calls init_worker from WorkerProc.__init__ (vllm/v1/executor/multiproc_executor.py:672). With one GPU, the default executor is uni (vllm/config/parallel.py:1099-1100). It calls init_worker inside the engine core process (vllm/v1/executor/uniproc_executor.py:68). There, the flag is already set, so nothing loads twice.

Model registry subprocess. Every built-in model is a lazy registry entry (vllm/model_executor/models/registry.py:1565-1573). An out-of-tree model is lazy too when you register it as a string "module:Class" (vllm/model_executor/models/registry.py:1187-1193). To read the capabilities of a lazy model without an import in the main process, vLLM may run a new Python process:

def _run() -> None:
    # Setup plugins
    from vllm.plugins import load_general_plugins

    load_general_plugins()

    fn, output_file = pickle.loads(sys.stdin.buffer.read())

    result = fn()

    with open(output_file, "wb") as f:
        f.write(pickle.dumps(result))

This happens only on a cache miss (vllm/model_executor/models/registry.py:1086-1109). A miss is a missing cache file, or a hash that does not match the model's source file. vLLM also skips the cache when it cannot find the source file. The command is python -m vllm.model_executor.models.registry (vllm/model_executor/models/registry.py:823). The page Out-of-tree models and the model registry explains this cache.

Step 6: platform plugins load on first use#

Platform plugins do not use load_general_plugins(). They load the first time that code in a process reads vllm.platforms.current_platform:

def __getattr__(name: str):
    if name == "current_platform":
        # lazy init current_platform.
        # 1. out-of-tree platform plugins need `from vllm.platforms import
        #    Platform` so that they can inherit `Platform` class. Therefore,
        #    we cannot resolve `current_platform` during the import of
        #    `vllm.platforms`.
        # 2. when users use out-of-tree platform plugins, they might run
        #    `import vllm`, some vllm internal code might access
        #    `current_platform` during the import, and we need to make sure
        #    `current_platform` is only resolved after the plugins are loaded
        #    (we have tests for this, if any developer violate this, they will
        #    see the test failures).
        global _current_platform
        if _current_platform is None:
            platform_cls_qualname = resolve_current_platform_cls_qualname()
            _current_platform = resolve_obj_by_qualname(platform_cls_qualname)()
            global _init_trace
            _init_trace = "".join(traceback.format_stack())
        return _current_platform

resolve_current_platform_cls_qualname() then loads the group vllm.platform_plugins (vllm/platforms/__init__.py:250). It calls each detector function, and one activated out-of-tree platform wins over the built-in platforms (vllm/platforms/__init__.py:277-279). If VLLM_TARGET_DEVICE is cpu, the function returns the CPU platform before it loads this group (vllm/platforms/__init__.py:244-248). The default value is cuda (vllm/envs.py:622). The page Platform plugins and current_platform explains the detection order.

Step 7: groups that only the front end loads#

• Stat loggers. AsyncLLM.__init__ adds the plugin classes to its stat loggers (vllm/v1/engine/async_llm.py:155-156). The group comment says "only when serve vLLM with async mode" (vllm/plugins/__init__.py:24-26).
• Endpoints. build_app attaches endpoint plugins after the core routes (vllm/entrypoints/launchers/app.py:47-51). These plugins load only when VLLM_PLUGINS is set and names them (vllm/plugins/__init__.py:122-131).
• IO processors. get_io_processor() loads the group only when it gets a plugin name (vllm/plugins/io_processors/__init__.py:42-61). The name comes from the io_processor_plugin setting of the model config, or from the same field in the Hugging Face config. The pooling serving code calls it (vllm/entrypoints/pooling/pooling/io_processor.py:51-55).

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

def load_plugins_by_group(group: str) -> dict[str, Callable[[], Any]]:
    """Load plugins registered under the given entry point group."""
    from importlib.metadata import entry_points

    allowed_plugins = envs.VLLM_PLUGINS

    discovered_plugins = entry_points(group=group)
    if len(discovered_plugins) == 0:
        logger.debug("No plugins for group %s found.", group)
        return {}

    # Check if the only discovered plugin is the default one
    is_default_group = group == DEFAULT_PLUGINS_GROUP
    # Use INFO for non-default groups and DEBUG for the default group
    log_level = logger.debug if is_default_group else logger.info

    log_level("Available plugins for group %s:", group)
    for plugin in discovered_plugins:
        log_level("- %s -> %s", plugin.name, plugin.value)

    if allowed_plugins is None:
        log_level(
            "All plugins in this group will be loaded. "
            "Set `VLLM_PLUGINS` to control which plugins to load."
        )

    plugins = dict[str, Callable[[], Any]]()
    for plugin in discovered_plugins:
        if allowed_plugins is None or plugin.name in allowed_plugins:
            if allowed_plugins is not None:
                log_level("Loading plugin %s", plugin.name)

            try:
                func = plugin.load()
                plugins[plugin.name] = func
            except Exception:
                logger.exception("Failed to load plugin %s", plugin.name)

    return plugins

What can go wrong#

• A plugin is missing, with no error. If the import of a plugin fails, vLLM writes "Failed to load plugin ..." with a traceback, and continues (vllm/plugins/__init__.py:71-72). Look for this line in the log of each process.
• Start-up stops in one process. An exception in a general plugin function stops the process that runs it (vllm/plugins/__init__.py:89-90). vLLM's own filesystem LoRA resolver raises ValueError when VLLM_LORA_RESOLVER_CACHE_DIR is set to a path that is not a folder (vllm/plugins/lora_resolvers/filesystem_resolver.py:51-57).
• Side effects run many times. A plugin function runs in each process (previous deep dive). Write it so that it can run again: check before you register, as the test plugin does.
• A second registration replaces the first. ModelRegistry.register_model overwrites an architecture that is already registered, with only a DEBUG log (vllm/model_executor/models/registry.py:1179-1185).
• CUDA starts too early. If a plugin imports model code that touches CUDA, the front end can initialize CUDA. Register models as "module:Class" strings. The docstring names the error that this avoids: "Cannot re-initialize CUDA in forked subprocess" (vllm/model_executor/models/registry.py:1167-1174).
• The docs say "every process". The design doc says that every process created by vLLM needs to load the plugin (docs/design/plugin_system.md:7). In the code, this is true for general and platform plugins only. Three groups load only in the front end (Step 7).

Why it matters. Every extension of vLLM starts with this loading step. To debug a plugin, you must know which process loads it and when. A model plugin that loads in the front end but fails in a worker gives an error only in the worker log. An allowlist that is too short gives no error at all.