Extension points
Code version: vLLM @ 3e44c0d · 2026-10-22
In one sentence
Each row names one place where out-of-tree code changes vLLM, the line that reads it, and the processes where your code must be visible.
This page is the lookup table. The guided version, with diagrams and a start-up timeline, is All the ways to extend vLLM. Read that page first if the three kinds of extension points are new to you. Every row here cites the code at the pinned commit; open the linked page for the explanation.
Words in the tables:
- Front end: the front end process of
vllm serve(API server,AsyncLLM). With more than one API server, each one is a front end. - Engine core: the EngineCore process, one per data-parallel rank.
- Workers: the worker processes. With TP = PP = DP = 1 there is no worker process: the worker runs inside the engine core process (
vllm/config/parallel.py:1099-1100). Then "workers" means the engine core process. - Registry subprocess: a short-lived process that inspects a model class. It runs only on a cache miss of the model-info cache, and it loads the general plugins first (
vllm/model_executor/models/registry.py:1608-1612).
The six entry-point groups#
vLLM finds these with importlib.metadata. Your package publishes an entry point in a plugin group; no other step is necessary. Five group names are constants in one file (vllm/plugins/__init__.py:16-30). The sixth, vllm.logits_processors, is in the sampler code of Model Runner V2 (vllm/v1/worker/gpu/sample/logits_processor/loader.py:34).
Loader and filter of each group#
| Group | Loader | Uses load_plugins_by_group | Filtered by VLLM_PLUGINS | On by default |
|---|---|---|---|---|
vllm.general_plugins | load_general_plugins (vllm/plugins/__init__.py:77-90) | yes (vllm/plugins/__init__.py:87) | yes | yes: unset VLLM_PLUGINS loads all (vllm/plugins/__init__.py:56-64) |
vllm.platform_plugins | resolve_current_platform_cls_qualname (vllm/platforms/__init__.py:250) | yes | yes | yes, but skipped when VLLM_TARGET_DEVICE=cpu (vllm/platforms/__init__.py:244-248) |
vllm.stat_logger_plugins | load_stat_logger_plugin_factories (vllm/v1/metrics/loggers.py:84-98) | yes (vllm/v1/metrics/loggers.py:87) | yes | yes |
vllm.endpoint_plugins | load_endpoint_plugins (vllm/plugins/__init__.py:93-159) | yes (vllm/plugins/__init__.py:133) | yes, and stricter: unset means none | no: only plugins named in VLLM_PLUGINS load; with the variable unset, vLLM logs a warning and loads none (vllm/plugins/__init__.py:122-131) |
vllm.io_processor_plugins | get_io_processor (vllm/plugins/io_processors/__init__.py:59-61) | yes | yes | only when the model asks for a plugin by name (vllm/plugins/io_processors/__init__.py:42-54) |
vllm.logits_processors | _load_v2_logitsprocs_plugins (vllm/v1/worker/gpu/sample/logits_processor/loader.py:37-74) | no: it calls entry_points() itself (vllm/v1/worker/gpu/sample/logits_processor/loader.py:46) | no | yes: every installed processor loads, for generative models |
Process, moment and errors of each group#
| Group | Your entry point gives | Processes | When in start-up | Import fails | Object fails or is wrong | Two entry points with one name | Page |
|---|---|---|---|---|---|---|---|
vllm.general_plugins | a function with no arguments | every process: front end (vllm/engine/arg_utils.py:3243), engine core (vllm/v1/engine/core.py:132), workers (vllm/v1/worker/worker_base.py:284), registry subprocess (vllm/model_executor/models/registry.py:1612) | front end: while the CLI builds its parsers; engine core: first line of EngineCore.__init__, before the executor (vllm/v1/engine/core.py:147); worker: start of init_worker. At most once per process (vllm/plugins/__init__.py:82-85) | logged, skipped (vllm/plugins/__init__.py:71-72) | not caught: the process stops (vllm/plugins/__init__.py:89-90) | the dict keeps one function per name (vllm/plugins/__init__.py:70; derived) | How vLLM finds and loads plugins |
vllm.platform_plugins | a detector that returns a Platform class name or None | every process | first use of current_platform (vllm/platforms/__init__.py:303-319); in every vllm serve process this is before the general plugins | logged, skipped | detector raises: DEBUG log, not active (vllm/platforms/__init__.py:255-265); two active out-of-tree platforms: RuntimeError (vllm/platforms/__init__.py:272-276) | same as general (derived) | Platform plugins and current_platform |
vllm.stat_logger_plugins | a StatLoggerBase subclass | front end | AsyncLLM.__init__ (vllm/v1/engine/async_llm.py:155-156), before it starts the engine core (vllm/v1/engine/async_llm.py:193) | logged, skipped | not a subclass: TypeError (vllm/v1/metrics/loggers.py:88-94) | same as general (derived) | Plugins in the API server |
vllm.endpoint_plugins | a factory that returns an EndpointPlugin | front end | build_app (vllm/entrypoints/launchers/app.py:51) | logged, skipped | factory raises: logged, skipped (vllm/plugins/__init__.py:137-141); required_tasks not served: INFO, skipped (vllm/plugins/__init__.py:143-154) | same as general (derived). Two plugins with the same route and method: the later one replaces the earlier route (vllm/plugins/endpoint_plugins/interface.py:121-139) | Plugins in the API server |
vllm.io_processor_plugins | a function that returns an IO processor class name or None | front end | pooling serving set-up (vllm/entrypoints/pooling/pooling/io_processor.py:51-55) | logged, skipped | function raises: WARNING (vllm/plugins/io_processors/__init__.py:70-71); none usable, or not the requested one: ValueError (vllm/plugins/io_processors/__init__.py:73-84) | same as general (derived) | Plugins in the API server |
vllm.logits_processors | a Model Runner V2 LogitsProcessor subclass | front end (request check, vllm/v1/engine/input_processor.py:58-62, vllm/v1/engine/input_processor.py:99-107) and workers (vllm/v1/worker/gpu/model_runner.py:463-466) | front end: InputProcessor.__init__; workers: load_model | RuntimeError (vllm/v1/worker/gpu/sample/logits_processor/loader.py:62-66) | not a V2 class: ValueError (vllm/v1/worker/gpu/sample/logits_processor/loader.py:67-72) | both load: the loader appends every entry point to a list (vllm/v1/worker/gpu/sample/logits_processor/loader.py:53-73) | Sampling and logits processors in Model Runner V2 |
Registries that a general plugin fills#
A registry is a module-level table plus a register function. The table lives in one process, so the register call must run in each process that reads the table. A general plugin does this, because every process runs it. Code in your own launch script reaches only that process. The table gives the read site, the processes that read, and what a second registration with the same name does.
| Extension point | Register with | Read at | Processes that read it | When | Same name again | Default | Page |
|---|---|---|---|---|---|---|---|
| Model architectures | ModelRegistry.register_model(arch, cls or "module:Class") (vllm/model_executor/models/registry.py:1160-1203) | inspect: vllm/config/model.py:741; build: vllm/model_executor/model_loader/utils.py:310 | front end (inspect), registry subprocess (on a cache miss), workers (build) | front end: while ModelConfig is made; workers: load_model | overwrites; DEBUG log (vllm/model_executor/models/registry.py:1179-1185) | opt-in; the built-in names are in _VLLM_MODELS (vllm/model_executor/models/registry.py:806-817) | Out-of-tree models and the model registry |
| Out-of-tree ops | @MyOp.register_oot or @CustomOp.register_oot(name=...) (vllm/model_executor/custom_op.py:335-356) | CustomOp.__new__ (vllm/model_executor/custom_op.py:107-126) | workers | when the model builds its layers | AssertionError "Duplicate op name" (vllm/model_executor/custom_op.py:338) | opt-in | CustomOp, out-of-tree ops and IR op providers |
| Out-of-tree layers | @MyLayer.register_oot on a PluggableLayer (vllm/model_executor/custom_op.py:82-99) | PluggableLayer.__new__ (vllm/model_executor/custom_op.py:56-65) | workers | when the model builds its layers | AssertionError "Duplicate layer name" (vllm/model_executor/custom_op.py:86) | opt-in | CustomOp, out-of-tree ops and IR op providers |
| Attention backends | register_backend(enum or "NAME", class_path) (vllm/v1/attention/backends/registry.py:348-420) | get_path (vllm/v1/attention/backends/registry.py:210) | workers; the front end parses the --attention-backend name (vllm/config/attention.py:216-219) | load_model, when each attention layer asks the selector | an enum member: the override is replaced, no log (vllm/v1/attention/backends/registry.py:413-418); a new string name that exists: ValueError (vllm/v1/attention/backends/registry.py:180-185) | opt-in | How vLLM chooses an attention backend |
| Quantization methods | @register_quantization_config("name") (vllm/model_executor/layers/quantization/__init__.py:61-109) | get_quantization_config (vllm/model_executor/layers/quantization/__init__.py:112-114) | front end; it makes the config object (vllm/config/vllm.py:1586-1589), and the workers get the object in the pickled VllmConfig | VllmConfig.__post_init__ | overwrites; DEBUG log (vllm/model_executor/layers/quantization/__init__.py:89-95, vllm/model_executor/layers/quantization/__init__.py:106) | opt-in | Quantization methods as plugins |
| Model loaders | @register_model_loader("format") (vllm/model_executor/model_loader/__init__.py:69-120) | get_model_loader (vllm/model_executor/model_loader/__init__.py:123-128) | workers | load_model | overwrites; WARNING log (vllm/model_executor/model_loader/__init__.py:101-107) | opt-in | Hooks in the engine core and the workers |
| KV connectors | KVConnectorFactory.register_connector(name, module, class) (vllm/distributed/kv_transfer/kv_connector/factory.py:31-40) | scheduler role vllm/v1/core/sched/scheduler.py:177-181; worker role vllm/distributed/kv_transfer/kv_transfer_state.py:88-92; class only vllm/config/vllm.py:1242, vllm/config/vllm.py:1966, vllm/config/vllm.py:2243, vllm/v1/metrics/loggers.py:132, vllm/distributed/kv_transfer/kv_connector/v1/metrics.py:161 | front end (class only), engine core, workers | front end: VllmConfig.__post_init__ and the stat loggers; engine core: when the scheduler is made; workers: KV cache set-up | ValueError (vllm/distributed/kv_transfer/kv_connector/factory.py:33-34); 17 built-in names exist (vllm/distributed/kv_transfer/kv_connector/factory.py:153-249) | opt-in (--kv-transfer-config) | KV connectors |
| Encoder-cache (EC) connectors | ECConnectorFactory.register_connector (vllm/distributed/ec_transfer/ec_connector/factory.py:23-33) | vllm/v1/core/sched/scheduler.py:203-205; vllm/distributed/ec_transfer/ec_transfer_state.py:37-38; metrics vllm/v1/metrics/loggers.py:134 | front end (metrics), engine core, workers | as for KV connectors | ValueError (vllm/distributed/ec_transfer/ec_connector/factory.py:26-27) | opt-in | KV connectors |
| KV offloading specs | OffloadingSpecFactory.register_spec (vllm/v1/kv_offload/factory.py:15-27), or spec_name + spec_module_path in the extra config (vllm/v1/kv_offload/factory.py:30-51) | create_spec in OffloadingConnector (vllm/distributed/kv_transfer/kv_connector/v1/offloading_connector.py:89) | where the offloading connector is made (engine core and workers; derived) | connector set-up | ValueError (vllm/v1/kv_offload/factory.py:18-19) | opt-in | KV connectors |
| Tool parsers | ToolParserManager.register_module (vllm/tool_parsers/abstract_tool_parser.py:327-370) | the --tool-call-parser name | front end | API server set-up and each request | module= form: overwrites when force=True (default), KeyError when force=False (vllm/tool_parsers/abstract_tool_parser.py:307-311); decorator form: replaces the lazy entry, no check (vllm/tool_parsers/abstract_tool_parser.py:364-366) | opt-in | Tool and reasoning parsers: from model text to tool_calls and reasoning |
| Reasoning parsers | ReasoningParserManager.register_module | the --reasoning-parser name | front end and engine core (structured outputs) | API server set-up; engine core when the structured-output manager is made (vllm/v1/structured_output/__init__.py:98-104) | same rule as the tool parser manager (vllm/reasoning/abs_reasoning_parsers.py:310-314) | opt-in | Tool and reasoning parsers: from model text to tool_calls and reasoning |
| LoRA resolvers | LoRAResolverRegistry.register_resolver (vllm/lora/resolver.py:52-72) | vllm/entrypoints/openai/models/serving.py:113-117 | front end | API server set-up; asked only with VLLM_ALLOW_RUNTIME_LORA_UPDATING=1 (vllm/entrypoints/serve/engine/serving.py:62-66) | overwrites; WARNING log (vllm/lora/resolver.py:64-70) | opt-in; the two built-in resolvers need their own variables | Plugins in the API server |
| Sleep-mode backends | SleepModeBackendFactory.register_backend (vllm/device_allocator/sleep_mode_backend.py:170-180) | vllm/v1/worker/gpu_worker.py:256-258 | workers | first sleep or wake call | ValueError (vllm/device_allocator/sleep_mode_backend.py:173-174) | opt-in; vllm serve has no flag for the backend name (cumem is used) | Hooks in the engine core and the workers |
| KV cache dtypes | @register_kv_cache_dtype("name") (vllm/config/cache.py:111-155) | KV_CACHE_DTYPES, STR_DTYPE_TO_TORCH_DTYPE, the handler (vllm/v1/kv_cache_interface.py:108-113) | every process that reads the dtype | config checks and KV cache set-up | overwrites the handler; WARNING log (vllm/config/cache.py:139-152) | opt-in; no caller in vllm/ | Hooks in the engine core and the workers |
Config hooks that name your class#
A config hook is a field of VllmConfig or a CLI flag that holds a class or module name. The front end parses it. The other processes get the string in the pickled VllmConfig and import the class themselves (vllm/v1/engine/utils.py:256-287). No general plugin is necessary. A field has one value, so there is no "duplicate".
| Field or flag | Defined at | Read at | Processes | When | Name format | Default | Page |
|---|---|---|---|---|---|---|---|
scheduler_cls (--scheduler-cls) | vllm/config/scheduler.py:177 | vllm/v1/engine/core.py:164; resolved at vllm/config/scheduler.py:231-256 | engine core | after the KV cache plan, before the scheduler is made | pkg.module.Class | None: AsyncScheduler or Scheduler (vllm/config/scheduler.py:232-239) | Hooks in the engine core and the workers |
worker_cls (--worker-cls) | vllm/config/parallel.py:306 | vllm/v1/worker/worker_base.py:292-296 | workers | init_worker, after the general plugins | pkg.module.Class | "auto": the platform sets it in check_and_update_config (vllm/platforms/cuda.py:330-331) | The Platform interface |
worker_extension_cls (--worker-extension-cls) | vllm/config/parallel.py:309 | vllm/v1/worker/worker_base.py:304-331 | workers | init_worker | pkg.module.Class | "" (none). A name that the worker already has: AssertionError (vllm/v1/worker/worker_base.py:315-319) | Hooks in the engine core and the workers |
logits_processors (--logits-processors) | vllm/config/model.py:407 | vllm/v1/worker/gpu/sample/logits_processor/loader.py:77-105 | front end (check) and workers | as for the entry-point group | pkg.module:Class, or a class | None | Sampling and logits processors in Model Runner V2 |
--tool-parser-plugin | vllm/entrypoints/common/cli_args.py:126 | vllm/entrypoints/common/cli_args.py:579-580 | front end | setup_server (vllm/entrypoints/launchers/launcher.py:369) and each API server worker (vllm/entrypoints/launchers/api_server/entry.py:183) | a module, or a path to a .py file; only values longer than 3 characters | "" | Tool and reasoning parsers: from model text to tool_calls and reasoning |
--reasoning-parser-plugin | vllm/config/structured_outputs.py:37 | vllm/entrypoints/common/cli_args.py:582-583; vllm/v1/structured_output/__init__.py:100-104 | front end and engine core | as above; engine core: structured-output manager | same as above | "" | Tool and reasoning parsers: from model text to tool_calls and reasoning |
kv_connector_module_path | vllm/config/kv_transfer.py:81 | vllm/distributed/kv_transfer/kv_connector/factory.py:103-124 | front end (class), engine core, workers | as for KV connectors | a module; the class name is kv_connector | None; when set, it wins over the registry (vllm/distributed/kv_transfer/kv_connector/factory.py:106-108) | KV connectors |
ec_connector_module_path | vllm/config/ec_transfer.py:54 | vllm/distributed/ec_transfer/ec_connector/factory.py:66-73 | engine core, workers | as for EC connectors | a module; the class name is ec_connector | None; the registry wins, this is the fallback | KV connectors |
model_class_overrides | vllm/config/model.py:351-357 | vllm/config/model.py:1166-1193 | front end and workers: each process registers the class once, on first use of model_config.registry | first model inspect or resolve | pkg.module:Class | empty; the docstring says "for development and debugging purposes only" | Out-of-tree models and the model registry |
Platform hooks#
A platform plugin returns a Platform subclass. vLLM calls its methods at fixed moments. These are extension points too, but only one platform is active in a process. The page The Platform interface explains each hook; the table gives the process and the moment.
| Hook | Called from | Processes | When | How often |
|---|---|---|---|---|
pre_register_and_update(parser) | vllm/engine/arg_utils.py:3257 | front end | each time the CLI builds a full serve parser: the subcommand parsers at start (vllm/entrypoints/cli/main.py:100-109), and again in setup_server through log_non_default_args (vllm/entrypoints/launchers/launcher.py:368, vllm/entrypoints/common/startup.py:70-71) | 5 times in the vllm serve front end: 4 with a parser, 1 without |
pre_register_and_update() | vllm/engine/arg_utils.py:2246 | front end | start of create_engine_config, after the flags are parsed | |
validate_environ | vllm/engine/arg_utils.py:2250 | front end | create_engine_config | once |
apply_config_platform_defaults | vllm/config/vllm.py:1804 | front end and every engine core | VllmConfig.__post_init__; the engine core runs __post_init__ again after the handshake (vllm/v1/engine/core.py:1336-1337) | once per process, plus each time code makes a new VllmConfig with replace() (vllm/config/utils.py:121-129), for example the EAGLE drafter in a worker (vllm/v1/worker/gpu/spec_decode/eagle/utils.py:90-91) |
check_and_update_config | vllm/config/vllm.py:2046 | front end and every engine core | late in VllmConfig.__post_init__ | |
register_triton_kernel_overrides | vllm/v1/worker/worker_base.py:290 | workers | init_worker, right after the general plugins | once |
get_device_communicator_cls | vllm/distributed/parallel_state.py:549-552 | workers | init_device | once per group with more than one rank |
get_attn_backend_cls | vllm/v1/attention/selector.py:218 | workers | load_model | once per distinct argument set (@cache) |
update_block_size_for_backend | vllm/v1/executor/multiproc_executor.py:702; vllm/v1/executor/uniproc_executor.py:75 | each worker (mp) or the engine core (uni) | right after load_model | once |
get_kv_cache_config_builder_cls | vllm/v1/core/kv_cache_config_builder.py:98-103 | engine core | KV cache planning | once |
import_kernels | vllm/_custom_ops.py:21 | each process that imports vllm._custom_ops | at that import | once per process |
How many times a general plugin runs#
A general plugin runs once in each process (the guard at vllm/plugins/__init__.py:82-85). So the count is the number of processes. Add 1 for each registry subprocess, which runs only on a cache miss of the model-info cache.
vllm serve with | Processes that run it | Runs |
|---|---|---|
| TP = 1 (default) | front end, engine core (the worker is inside it) | 2 |
| TP = 2 | front end, engine core, 2 workers | 4 |
| TP = 4 | front end, engine core, 4 workers | 6 |
The process layout comes from the executor choice: mp when the world size across DP is more than 1, else uni (vllm/config/parallel.py:1055, vllm/config/parallel.py:1099-1100). The page Executors and workers: from the engine core to the GPUs explains it.
Name formats#
| Where | Format | Parsed by |
|---|---|---|
platform detector result, worker_cls, worker_extension_cls, scheduler_cls, register_backend(class_path=...) | pkg.module.Class | resolve_obj_by_qualname, split at the last dot (vllm/utils/import_utils.py:119-123) |
register_model, logits_processors, model_class_overrides | pkg.module:Class | split at ":"; another count raises ValueError (vllm/model_executor/models/registry.py:1187-1193, vllm/v1/worker/gpu/sample/logits_processor/loader.py:99-104) |
| KV, EC and offloading factories | module path and class name as two strings | importlib.import_module + getattr (vllm/distributed/kv_transfer/kv_connector/factory.py:36-38) |
register_oot(name=...) | the in-tree class name, for example "RMSNorm" | __new__ looks up cls.__name__ (vllm/model_executor/custom_op.py:107-118) |
The FAQ lists the places where the docs show another form: FAQ.
Check yourself#
- 1Which of the six entry-point groups ignoreseasy
VLLM_PLUGINS, and which one loads nothing whenVLLM_PLUGINSis unset?Show answer
vllm.logits_processorsignores it, because its loader callsentry_points()directly (vllm/v1/worker/gpu/sample/logits_processor/loader.py:46).vllm.endpoint_pluginsloads nothing when the variable is unset (vllm/plugins/__init__.py:122-131). - 2Your platform'smedium
check_and_update_configappends"+my_op"to the custom-op list. You serve with TP = 2. How many times does the hook run at start-up, and what can go wrong?Show answer
At least twice: once in the front end (
vllm/config/vllm.py:2046) and once in the engine core, which runs__post_init__again (vllm/v1/engine/core.py:1336-1337). The engine core gets the config that the front end already changed, so a hook that appends without a check adds the entry a second time. Make the hook give the same result when it runs again. - 3A general plugin callshard
register_quantization_config("fp8")andregister_model_loader("auto"). Neither raises. What happens to the built-in entries, and what does the log show at INFO?Show answer
Both registries overwrite. The quantization registry logs at DEBUG only (
vllm/model_executor/layers/quantization/__init__.py:89-95), so INFO shows nothing for it. The model loader registry logs a WARNING (vllm/model_executor/model_loader/__init__.py:101-107) and then an INFO line "Registered model loader" (vllm/model_executor/model_loader/__init__.py:113-117). The general group's own list of plugins is at DEBUG (vllm/plugins/__init__.py:48-54).