distributed/kv_transfer/kv_connector/factory.py: the KV connector registry
Code version: vLLM @ 3e44c0d · 2026-10-22
Purpose#
vllm/distributed/kv_transfer/kv_connector/factory.py turns the connector name in --kv-transfer-config into a KV connector class, and makes one object of it in a given connector role.
| File | vllm/distributed/kv_transfer/kv_connector/factory.py, 249 lines of Python. One class with five class methods, and 17 module-level register_connector calls. |
|---|---|
| Main class | KVConnectorFactory (vllm/distributed/kv_transfer/kv_connector/factory.py:27) |
| Runs in | Every process that imports it, on the CPU: the front end (class lookups for config checks and metrics), the EngineCore process (the scheduler-side object) and each worker (the worker-side object). |
| Called by | The scheduler constructor, ensure_kv_transfer_initialized in the worker, VllmConfig checks, the KV connector metrics classes, MultiConnector and a layout helper (see "Who calls it"). |
| Calls | importlib.import_module, supports_hma (vllm/distributed/kv_transfer/kv_connector/factory.py:13-16), supports_kw (vllm/distributed/kv_transfer/kv_connector/factory.py:18) and the connector constructor. |
| Not in this file | The base class KVConnectorBase_V1 and the role enum (kv_connector/v1/base.py), the connectors themselves (kv_connector/v1/), the config (vllm/config/kv_transfer.py), the EC connector factory (vllm/distributed/ec_transfer/ec_connector/factory.py). |
The page KV connectors explains connectors from the reader's side. This page reads the factory from top to bottom.
Where it sits#
The file has three parts. The class body has the registry and the lookups (lines 27-146). After it, 17 calls register the built-in connectors (lines 149-249). A comment says why they live here and not in each connector file: "we want to only load the files corresponding to the current connector" (lines 149-151).
Who calls it#
- The scheduler makes the
SCHEDULERobject in its constructor (vllm/v1/core/sched/scheduler.py:177-181). - Each worker makes the
WORKERobject once, inensure_kv_transfer_initialized(vllm/distributed/kv_transfer/kv_transfer_state.py:73-92), called fromWorkerafter the KV cache config is known (vllm/v1/worker/gpu_worker.py:873-878). VllmConfigin the front end reads the class for three checks: HMA support (vllm/config/vllm.py:2243), sleep mode (vllm/config/vllm.py:1242-1249) and PIECEWISE CUDA graphs (vllm/config/vllm.py:1966-1980).- Metrics.
KVConnectorLoggingand the Prometheus metrics class read the class (vllm/distributed/kv_transfer/kv_connector/v1/metrics.py:57,vllm/distributed/kv_transfer/kv_connector/v1/metrics.py:161); the stat logger makes the first one in the front end (vllm/v1/metrics/loggers.py:132). - Other code. A layout helper asks for the required KV cache layout (
vllm/distributed/kv_transfer/kv_connector/utils.py:56-57).MultiConnectorresolves its children (vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:150,vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:168,vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:694). - Tests and plugins call
register_connector, for exampletests/v1/kv_connector/unit/test_multi_connector.py:146.
What it calls#
importlib.import_modulein the loader and for the module path (vllm/distributed/kv_transfer/kv_connector/factory.py:37,vllm/distributed/kv_transfer/kv_connector/factory.py:108).supports_kw(connector_cls, "kv_cache_config")to reject old constructors (vllm/distributed/kv_transfer/kv_connector/factory.py:116).supports_hma, anissubclasscheck againstSupportsHMA(vllm/distributed/kv_transfer/kv_connector/v1/base.py:132-136), andMultiConnector.all_children_support_hma(vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:156-170).- The connector constructor
connector_cls(config, role, kv_cache_config)(line 75).
Key classes and functions#
| Name | Line | What it does |
|---|---|---|
KVConnectorFactory._registry | vllm/distributed/kv_transfer/kv_connector/factory.py:28 | dict[str, Callable[[], type]]: name to a function that imports the class. |
register_connector(name, module_path, class_name) | vllm/distributed/kv_transfer/kv_connector/factory.py:30-40 | Stores a lazy loader; a taken name raises ValueError. |
create_connector(config, role, kv_cache_config) | vllm/distributed/kv_transfer/kv_connector/factory.py:42-75 | Finds the class, checks HMA, logs, makes the object. |
get_connector_class_by_name(name) | vllm/distributed/kv_transfer/kv_connector/factory.py:77-94 | Registry only; an unknown name raises ValueError. |
get_connector_class(kv_transfer_config) | vllm/distributed/kv_transfer/kv_connector/factory.py:96-129 | Module path first, then registry, else ValueError. |
supports_hma_config(kv_transfer_config) | vllm/distributed/kv_transfer/kv_connector/factory.py:131-146 | True if the class supports the hybrid KV cache manager; for MultiConnector, all children must. |
17 register_connector calls | vllm/distributed/kv_transfer/kv_connector/factory.py:153-249 | The built-in names, from ExampleConnector to HF3FSKVConnector. |
Walk through the main methods#
register_connector: a lazy entry#
class KVConnectorFactory:
_registry: dict[str, Callable[[], type[KVConnectorBase]]] = {}
@classmethod
def register_connector(cls, name: str, module_path: str, class_name: str) -> None:
"""Register a connector with a lazy-loading module and class name."""
if name in cls._registry:
raise ValueError(f"Connector '{name}' is already registered.")
def loader() -> type[KVConnectorBase]:
module = importlib.import_module(module_path)
return getattr(module, class_name)
cls._registry[name] = loader- Line 28. The registry is a class attribute, so there is one per process.
- Lines 33-34. A second registration of any name raises. There is no override form.
- Lines 36-38. The loader is a closure. It imports the module only when it is called, so the front end can register NIXL or Mooncake without importing their libraries.
get_connector_class: module path wins#
@classmethod
def get_connector_class(
cls, kv_transfer_config: "KVTransferConfig"
) -> type[KVConnectorBaseType]:
connector_name = kv_transfer_config.kv_connector
if connector_name is None:
raise ValueError("Connector name is not set in KVTransferConfig")
connector_module_path = kv_transfer_config.kv_connector_module_path
if connector_module_path is not None and not connector_module_path:
raise ValueError("kv_connector_module_path cannot be an empty string.")
if connector_module_path:
# External module path takes priority over internal registry.
connector_module = importlib.import_module(connector_module_path)
try:
connector_cls = getattr(connector_module, connector_name)
except AttributeError as e:
raise AttributeError(
f"Class {connector_name} not found in {connector_module_path}"
) from e
connector_cls = cast(type[KVConnectorBaseType], connector_cls)
if not supports_kw(connector_cls, "kv_cache_config"):
msg = (
f"Connector {connector_cls.__name__} uses deprecated "
"2-argument constructor signature. External v1 KV "
"connectors must accept kv_cache_config as the third "
"constructor argument and pass it to super().__init__()."
)
logger.error(msg)
raise ValueError(msg)
elif connector_name in cls._registry:
connector_cls = cls._registry[connector_name]()
else:
raise ValueError(f"Unsupported connector type: {connector_name}")
return connector_cls- Lines 100-102. No
kv_connector:ValueError. - Lines 103-105. An empty string as module path is an error;
Nonemeans "not set". - Lines 106-114. With a module path,
kv_connectoris the class name inside that module. A missing class raisesAttributeErrorwith the module path. - Lines 116-124. The constructor must accept
kv_cache_config. The message says that the "2-argument constructor signature" is deprecated; vLLM logs it at ERROR level and raises. - Lines 125-128. Without a module path, the registry; else "Unsupported connector type".
The EC connector factory uses the opposite order: registry first, module path as the fallback (vllm/distributed/ec_transfer/ec_connector/factory.py:66-73).
create_connector: one role, one object#
@classmethod
def create_connector(
cls,
config: "VllmConfig",
role: KVConnectorRole,
kv_cache_config: "KVCacheConfig",
) -> KVConnectorBase:
kv_transfer_config = config.kv_transfer_config
if kv_transfer_config is None:
raise ValueError("kv_transfer_config must be set to create a connector")
connector_cls = cls.get_connector_class(kv_transfer_config)
# check if the connector supports HMA
hma_enabled = not config.scheduler_config.disable_hybrid_kv_cache_manager
if hma_enabled and not cls.supports_hma_config(kv_transfer_config):
raise ValueError(
f"Connector {connector_cls.__name__} does not support HMA but "
f"HMA is enabled. Please set `--disable-hybrid-kv-cache-manager`."
)
logger.info(
"Creating v1 connector with name: %s and engine_id: %s",
connector_cls.__name__,
kv_transfer_config.engine_id,
)
# NOTE(Kuntai): v1 connector is explicitly separated into two roles.
# Scheduler connector:
# - Co-locate with scheduler process
# - Should only be used inside the Scheduler class
# Worker connector:
# - Co-locate with worker process
# - Should only be used inside the forward context & attention layer
# We build separately to enforce strict separation
return connector_cls(config, role, kv_cache_config)- Lines 49-52. The config must have a KV transfer config; the class comes from
get_connector_class. - Lines 55-60. If the hybrid KV cache manager is on and the connector does not support it,
ValueErrorasks for--disable-hybrid-kv-cache-manager.VllmConfignormally turns the manager off first (vllm/config/vllm.py:2236-2259), so this raises only when the user forced the manager on. - Lines 62-66. An INFO log names the class and the engine id.
- Lines 67-75. The comment explains the two roles: the scheduler connector "should only be used inside the Scheduler class", the worker connector "inside the forward context & attention layer". The same class is built with the role as an argument.
supports_hma_config: the MultiConnector case#
@classmethod
def supports_hma_config(cls, kv_transfer_config: "KVTransferConfig") -> bool:
"""Return whether this KV transfer config supports HMA.
MultiConnector is a special case: the wrapper class implements
SupportsHMA, but effective support depends on every configured child.
"""
connector_cls = cls.get_connector_class(kv_transfer_config)
if kv_transfer_config.kv_connector != "MultiConnector":
return supports_hma(connector_cls)
from vllm.distributed.kv_transfer.kv_connector.v1.multi_connector import (
MultiConnector,
)
return MultiConnector.all_children_support_hma(kv_transfer_config)A normal connector supports HMA when its class subclasses SupportsHMA. MultiConnector subclasses it too, but the docstring says that "effective support depends on every configured child". So the factory asks all_children_support_hma, which returns False for an empty child list.
Deep dive#
The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.
Deep diveData flow: types in and out
| Value | Type | Where |
|---|---|---|
_registry | dict[str, Callable[[], type[KVConnectorBase]]], 17 entries after import | vllm/distributed/kv_transfer/kv_connector/factory.py:28 |
kv_transfer_config.kv_connector | str | None: a registry name, or a class name inside the module path | vllm/config/kv_transfer.py:60 |
kv_transfer_config.kv_connector_module_path | str | None: an importable module name (dots, not a file path) | vllm/config/kv_transfer.py:81 |
role | KVConnectorRole.SCHEDULER (0) or WORKER (1) | vllm/distributed/kv_transfer/kv_connector/v1/base.py:139-144 |
kv_cache_config | KVCacheConfig: the planned KV cache groups and blocks | vllm/distributed/kv_transfer/kv_connector/factory.py:47 |
return of create_connector | a KVConnectorBase object | vllm/distributed/kv_transfer/kv_connector/factory.py:48 |
KVConnectorBase and KVConnectorBaseType come from kv_connector/base.py (vllm/distributed/kv_transfer/kv_connector/factory.py:9-12).
Deep diveProcesses and import time
The 17 built-in names enter the registry when Python imports the module, in each process that imports it. For vllm serve, the front end imports it at start-up: the CLI module imports the multiprocess executor, which imports the connector utilities, which import the factory (vllm/entrypoints/cli/serve.py:29, vllm/v1/executor/multiproc_executor.py:33, vllm/distributed/kv_transfer/kv_connector/utils.py:17). The imports inside VllmConfig are lazy (vllm/config/vllm.py:2239-2241). The scheduler, the worker-side state module and the metrics module import it at module level (vllm/v1/core/sched/scheduler.py:21, vllm/distributed/kv_transfer/kv_transfer_state.py:6, vllm/distributed/kv_transfer/kv_connector/v1/metrics.py:9).
A plugin's register_connector must therefore run after the factory module is imported and before the lookup. A general plugin that imports KVConnectorFactory and calls register_connector satisfies both: its own import loads the built-ins first. It runs in the front end, the engine core and each worker (How vLLM finds and loads plugins).
With TP = 2 one vllm serve makes 3 connector objects: 1 SCHEDULER in the engine core and 1 WORKER in each of the 2 workers. The front end makes none; it only gets the class. (Derived from "Who calls it".)
Deep diveWorked example: a connector from a module path
You write acme_kv/connector.py with class AcmeConnector(KVConnectorBase_V1) and start:
vllm serve Qwen/Qwen3-8B --kv-transfer-config \
'{"kv_connector":"AcmeConnector","kv_role":"kv_both","kv_connector_module_path":"acme_kv.connector"}'
- The front end checks HMA:
get_connector_classimportsacme_kv.connectorand takesAcmeConnector(lines 106-110). It does not subclassSupportsHMA, soVllmConfigturns the hybrid manager off with a WARNING (vllm/config/vllm.py:2243-2256). supports_kwfindskv_cache_configin the constructor (line 116); the base class has it (vllm/distributed/kv_transfer/kv_connector/v1/base.py:214-219).- The engine core imports the module again and makes
AcmeConnector(config, KVConnectorRole.SCHEDULER, kv_cache_config). - Each worker does the same with
WORKER.
No registration is needed, because every lookup imports the module itself. An offline example in the repository uses the same field (examples/disaggregated/kv_load_failure_recovery_offline/decode_example.py:44).
Deep diveError paths: what raises, what only logs
get_connector_class_by_name reads only the registry (vllm/distributed/kv_transfer/kv_connector/factory.py:92-94). Its one caller in vllm/ rebuilds the stats of MultiConnector children by name, when the stats arrive as a dict and not as a stats object (vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:686-696). The name is the child's class name (vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:716). A module-path child is not in the registry. Whether its stats reach line 694 as a dict is not traced here (open question).
Deep diveEnvironment variables and flags
The factory reads no environment variable. Its inputs come from --kv-transfer-config (vllm/engine/arg_utils.py:1871), and indirectly from --kv-offloading-size, which fills kv_connector with OffloadingConnector or SimpleCPUOffloadConnector (vllm/config/vllm.py:1149-1157). VLLM_USE_SIMPLE_KV_OFFLOAD (default 0) picks between the two (vllm/envs.py:2239-2241).
Related concepts#
- KV connectors: the connector interface, the two roles and the scheduler integration.
- All the ways to extend vLLM: all registries, and what a second registration does in each.
- Disaggregated prefill: prefill and decode on different instances: the main use of connectors between instances.
- The hybrid KV cache: specs, groups and one manager per layer type: the hybrid KV cache manager that
supports_hma_configprotects. - v1/core/sched/scheduler.py: the Scheduler: the scheduler that owns the
SCHEDULERobject.
Check yourself#
- 1What does the registry store for each name: a class, an object or something else?easy
Show answer
A function with no arguments that imports the module and returns the class (
vllm/distributed/kv_transfer/kv_connector/factory.py:36-40). The module is imported only at the first lookup. - 2With a module path set, what mustmedium
kv_connectorcontain?Show answer
The class name inside that module:
getattr(connector_module, connector_name)(vllm/distributed/kv_transfer/kv_connector/factory.py:108-114). The registry is not used. - 3Why canmedium
create_connectorraise for a hybrid model even thoughVllmConfigchecks HMA first?Show answer
VllmConfigturns the manager off only whendisable_hybrid_kv_cache_managerisNone(vllm/config/vllm.py:2236-2259). If the user forced the manager on,create_connectorstill finds HMA on and raises (vllm/distributed/kv_transfer/kv_connector/factory.py:55-60). - 4You serve with TP = 4 and a connector registered by a general plugin. How many times doeshard
register_connectorrun, and how many connector objects exist?Show answer
The general plugin runs once in each process: front end, engine core and 4 workers, so 6 registrations, one per process registry (
vllm/plugins/__init__.py:82-90). Objects: 1SCHEDULERin the engine core (vllm/v1/core/sched/scheduler.py:177-181) and 1WORKERper worker (vllm/distributed/kv_transfer/kv_transfer_state.py:88-92), so 5. The front end only reads the class. (A model registry subprocess, if one runs, adds one more registration.)