vLLM Explained

distributed/kv_transfer/kv_connector/factory.py: the KV connector registry

AdvancedCode mapabout 7 min read + 4 min deep dives

Code version: vLLM @ 3e44c0d · 2026-10-22

Purpose#

vllm/distributed/kv_transfer/kv_connector/factory.py turns the connector name in --kv-transfer-config into a KV connector class, and makes one object of it in a given connector role.

Filevllm/distributed/kv_transfer/kv_connector/factory.py, 249 lines of Python. One class with five class methods, and 17 module-level register_connector calls.
Main classKVConnectorFactory (vllm/distributed/kv_transfer/kv_connector/factory.py:27)
Runs inEvery process that imports it, on the CPU: the front end (class lookups for config checks and metrics), the EngineCore process (the scheduler-side object) and each worker (the worker-side object).
Called byThe scheduler constructor, ensure_kv_transfer_initialized in the worker, VllmConfig checks, the KV connector metrics classes, MultiConnector and a layout helper (see "Who calls it").
Callsimportlib.import_module, supports_hma (vllm/distributed/kv_transfer/kv_connector/factory.py:13-16), supports_kw (vllm/distributed/kv_transfer/kv_connector/factory.py:18) and the connector constructor.
Not in this fileThe base class KVConnectorBase_V1 and the role enum (kv_connector/v1/base.py), the connectors themselves (kv_connector/v1/), the config (vllm/config/kv_transfer.py), the EC connector factory (vllm/distributed/ec_transfer/ec_connector/factory.py).

The page KV connectors explains connectors from the reader's side. This page reads the factory from top to bottom.

Where it sits#

Figure: the lookup order of get_connector_class (panel 1) and the three kinds of callers (panel 2). Line numbers without a file are in factory.py.
Open full size ↗

The file has three parts. The class body has the registry and the lookups (lines 27-146). After it, 17 calls register the built-in connectors (lines 149-249). A comment says why they live here and not in each connector file: "we want to only load the files corresponding to the current connector" (lines 149-151).

Who calls it#

What it calls#

Key classes and functions#

NameLineWhat it does
KVConnectorFactory._registryvllm/distributed/kv_transfer/kv_connector/factory.py:28dict[str, Callable[[], type]]: name to a function that imports the class.
register_connector(name, module_path, class_name)vllm/distributed/kv_transfer/kv_connector/factory.py:30-40Stores a lazy loader; a taken name raises ValueError.
create_connector(config, role, kv_cache_config)vllm/distributed/kv_transfer/kv_connector/factory.py:42-75Finds the class, checks HMA, logs, makes the object.
get_connector_class_by_name(name)vllm/distributed/kv_transfer/kv_connector/factory.py:77-94Registry only; an unknown name raises ValueError.
get_connector_class(kv_transfer_config)vllm/distributed/kv_transfer/kv_connector/factory.py:96-129Module path first, then registry, else ValueError.
supports_hma_config(kv_transfer_config)vllm/distributed/kv_transfer/kv_connector/factory.py:131-146True if the class supports the hybrid KV cache manager; for MultiConnector, all children must.
17 register_connector callsvllm/distributed/kv_transfer/kv_connector/factory.py:153-249The built-in names, from ExampleConnector to HF3FSKVConnector.

Walk through the main methods#

register_connector: a lazy entry#

class KVConnectorFactory:
    _registry: dict[str, Callable[[], type[KVConnectorBase]]] = {}

    @classmethod
    def register_connector(cls, name: str, module_path: str, class_name: str) -> None:
        """Register a connector with a lazy-loading module and class name."""
        if name in cls._registry:
            raise ValueError(f"Connector '{name}' is already registered.")

        def loader() -> type[KVConnectorBase]:
            module = importlib.import_module(module_path)
            return getattr(module, class_name)

        cls._registry[name] = loader
  1. Line 28. The registry is a class attribute, so there is one per process.
  2. Lines 33-34. A second registration of any name raises. There is no override form.
  3. Lines 36-38. The loader is a closure. It imports the module only when it is called, so the front end can register NIXL or Mooncake without importing their libraries.

get_connector_class: module path wins#

    @classmethod
    def get_connector_class(
        cls, kv_transfer_config: "KVTransferConfig"
    ) -> type[KVConnectorBaseType]:
        connector_name = kv_transfer_config.kv_connector
        if connector_name is None:
            raise ValueError("Connector name is not set in KVTransferConfig")
        connector_module_path = kv_transfer_config.kv_connector_module_path
        if connector_module_path is not None and not connector_module_path:
            raise ValueError("kv_connector_module_path cannot be an empty string.")
        if connector_module_path:
            # External module path takes priority over internal registry.
            connector_module = importlib.import_module(connector_module_path)
            try:
                connector_cls = getattr(connector_module, connector_name)
            except AttributeError as e:
                raise AttributeError(
                    f"Class {connector_name} not found in {connector_module_path}"
                ) from e
            connector_cls = cast(type[KVConnectorBaseType], connector_cls)
            if not supports_kw(connector_cls, "kv_cache_config"):
                msg = (
                    f"Connector {connector_cls.__name__} uses deprecated "
                    "2-argument constructor signature. External v1 KV "
                    "connectors must accept kv_cache_config as the third "
                    "constructor argument and pass it to super().__init__()."
                )
                logger.error(msg)
                raise ValueError(msg)
        elif connector_name in cls._registry:
            connector_cls = cls._registry[connector_name]()
        else:
            raise ValueError(f"Unsupported connector type: {connector_name}")
        return connector_cls
  1. Lines 100-102. No kv_connector: ValueError.
  2. Lines 103-105. An empty string as module path is an error; None means "not set".
  3. Lines 106-114. With a module path, kv_connector is the class name inside that module. A missing class raises AttributeError with the module path.
  4. Lines 116-124. The constructor must accept kv_cache_config. The message says that the "2-argument constructor signature" is deprecated; vLLM logs it at ERROR level and raises.
  5. Lines 125-128. Without a module path, the registry; else "Unsupported connector type".

The EC connector factory uses the opposite order: registry first, module path as the fallback (vllm/distributed/ec_transfer/ec_connector/factory.py:66-73).

create_connector: one role, one object#

    @classmethod
    def create_connector(
        cls,
        config: "VllmConfig",
        role: KVConnectorRole,
        kv_cache_config: "KVCacheConfig",
    ) -> KVConnectorBase:
        kv_transfer_config = config.kv_transfer_config
        if kv_transfer_config is None:
            raise ValueError("kv_transfer_config must be set to create a connector")
        connector_cls = cls.get_connector_class(kv_transfer_config)

        # check if the connector supports HMA
        hma_enabled = not config.scheduler_config.disable_hybrid_kv_cache_manager
        if hma_enabled and not cls.supports_hma_config(kv_transfer_config):
            raise ValueError(
                f"Connector {connector_cls.__name__} does not support HMA but "
                f"HMA is enabled. Please set `--disable-hybrid-kv-cache-manager`."
            )

        logger.info(
            "Creating v1 connector with name: %s and engine_id: %s",
            connector_cls.__name__,
            kv_transfer_config.engine_id,
        )
        # NOTE(Kuntai): v1 connector is explicitly separated into two roles.
        # Scheduler connector:
        # - Co-locate with scheduler process
        # - Should only be used inside the Scheduler class
        # Worker connector:
        # - Co-locate with worker process
        # - Should only be used inside the forward context & attention layer
        # We build separately to enforce strict separation
        return connector_cls(config, role, kv_cache_config)
  1. Lines 49-52. The config must have a KV transfer config; the class comes from get_connector_class.
  2. Lines 55-60. If the hybrid KV cache manager is on and the connector does not support it, ValueError asks for --disable-hybrid-kv-cache-manager. VllmConfig normally turns the manager off first (vllm/config/vllm.py:2236-2259), so this raises only when the user forced the manager on.
  3. Lines 62-66. An INFO log names the class and the engine id.
  4. Lines 67-75. The comment explains the two roles: the scheduler connector "should only be used inside the Scheduler class", the worker connector "inside the forward context & attention layer". The same class is built with the role as an argument.

supports_hma_config: the MultiConnector case#

    @classmethod
    def supports_hma_config(cls, kv_transfer_config: "KVTransferConfig") -> bool:
        """Return whether this KV transfer config supports HMA.

        MultiConnector is a special case: the wrapper class implements
        SupportsHMA, but effective support depends on every configured child.
        """
        connector_cls = cls.get_connector_class(kv_transfer_config)
        if kv_transfer_config.kv_connector != "MultiConnector":
            return supports_hma(connector_cls)

        from vllm.distributed.kv_transfer.kv_connector.v1.multi_connector import (
            MultiConnector,
        )

        return MultiConnector.all_children_support_hma(kv_transfer_config)

A normal connector supports HMA when its class subclasses SupportsHMA. MultiConnector subclasses it too, but the docstring says that "effective support depends on every configured child". So the factory asks all_children_support_hma, which returns False for an empty child list.

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

Deep dive

Data flow: types in and out

ValueTypeWhere
_registrydict[str, Callable[[], type[KVConnectorBase]]], 17 entries after importvllm/distributed/kv_transfer/kv_connector/factory.py:28
kv_transfer_config.kv_connectorstr | None: a registry name, or a class name inside the module pathvllm/config/kv_transfer.py:60
kv_transfer_config.kv_connector_module_pathstr | None: an importable module name (dots, not a file path)vllm/config/kv_transfer.py:81
roleKVConnectorRole.SCHEDULER (0) or WORKER (1)vllm/distributed/kv_transfer/kv_connector/v1/base.py:139-144
kv_cache_configKVCacheConfig: the planned KV cache groups and blocksvllm/distributed/kv_transfer/kv_connector/factory.py:47
return of create_connectora KVConnectorBase objectvllm/distributed/kv_transfer/kv_connector/factory.py:48

KVConnectorBase and KVConnectorBaseType come from kv_connector/base.py (vllm/distributed/kv_transfer/kv_connector/factory.py:9-12).

Deep dive

Processes and import time

The 17 built-in names enter the registry when Python imports the module, in each process that imports it. For vllm serve, the front end imports it at start-up: the CLI module imports the multiprocess executor, which imports the connector utilities, which import the factory (vllm/entrypoints/cli/serve.py:29, vllm/v1/executor/multiproc_executor.py:33, vllm/distributed/kv_transfer/kv_connector/utils.py:17). The imports inside VllmConfig are lazy (vllm/config/vllm.py:2239-2241). The scheduler, the worker-side state module and the metrics module import it at module level (vllm/v1/core/sched/scheduler.py:21, vllm/distributed/kv_transfer/kv_transfer_state.py:6, vllm/distributed/kv_transfer/kv_connector/v1/metrics.py:9).

A plugin's register_connector must therefore run after the factory module is imported and before the lookup. A general plugin that imports KVConnectorFactory and calls register_connector satisfies both: its own import loads the built-ins first. It runs in the front end, the engine core and each worker (How vLLM finds and loads plugins).

With TP = 2 one vllm serve makes 3 connector objects: 1 SCHEDULER in the engine core and 1 WORKER in each of the 2 workers. The front end makes none; it only gets the class. (Derived from "Who calls it".)

Deep dive

Worked example: a connector from a module path

You write acme_kv/connector.py with class AcmeConnector(KVConnectorBase_V1) and start:

vllm serve Qwen/Qwen3-8B --kv-transfer-config \
  '{"kv_connector":"AcmeConnector","kv_role":"kv_both","kv_connector_module_path":"acme_kv.connector"}'
  1. The front end checks HMA: get_connector_class imports acme_kv.connector and takes AcmeConnector (lines 106-110). It does not subclass SupportsHMA, so VllmConfig turns the hybrid manager off with a WARNING (vllm/config/vllm.py:2243-2256).
  2. supports_kw finds kv_cache_config in the constructor (line 116); the base class has it (vllm/distributed/kv_transfer/kv_connector/v1/base.py:214-219).
  3. The engine core imports the module again and makes AcmeConnector(config, KVConnectorRole.SCHEDULER, kv_cache_config).
  4. Each worker does the same with WORKER.

No registration is needed, because every lookup imports the module itself. An offline example in the repository uses the same field (examples/disaggregated/kv_load_failure_recovery_offline/decode_example.py:44).

Deep dive

Error paths: what raises, what only logs

SituationResultWhere
Name registered twiceValueError "Connector '...' is already registered."vllm/distributed/kv_transfer/kv_connector/factory.py:33-34
No KV transfer config in create_connectorValueErrorvllm/distributed/kv_transfer/kv_connector/factory.py:50-51
kv_connector is NoneValueErrorvllm/distributed/kv_transfer/kv_connector/factory.py:101-102
Module path is ""ValueErrorvllm/distributed/kv_transfer/kv_connector/factory.py:104-105
Module path does not importthe import error, not caughtvllm/distributed/kv_transfer/kv_connector/factory.py:108
Class missing in the moduleAttributeErrorvllm/distributed/kv_transfer/kv_connector/factory.py:111-114
Old 2-argument constructorERROR log, then ValueErrorvllm/distributed/kv_transfer/kv_connector/factory.py:116-124
Unknown name, no module pathValueError "Unsupported connector type"vllm/distributed/kv_transfer/kv_connector/factory.py:127-128
HMA on, connector without HMAValueErrorvllm/distributed/kv_transfer/kv_connector/factory.py:55-60
A registered module fails to importthe import error, at the first lookupvllm/distributed/kv_transfer/kv_connector/factory.py:36-38

get_connector_class_by_name reads only the registry (vllm/distributed/kv_transfer/kv_connector/factory.py:92-94). Its one caller in vllm/ rebuilds the stats of MultiConnector children by name, when the stats arrive as a dict and not as a stats object (vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:686-696). The name is the child's class name (vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py:716). A module-path child is not in the registry. Whether its stats reach line 694 as a dict is not traced here (open question).

Deep dive

Environment variables and flags

The factory reads no environment variable. Its inputs come from --kv-transfer-config (vllm/engine/arg_utils.py:1871), and indirectly from --kv-offloading-size, which fills kv_connector with OffloadingConnector or SimpleCPUOffloadConnector (vllm/config/vllm.py:1149-1157). VLLM_USE_SIMPLE_KV_OFFLOAD (default 0) picks between the two (vllm/envs.py:2239-2241).

Check yourself#

  1. 1
    What does the registry store for each name: a class, an object or something else?
    easy
    Show answer

    A function with no arguments that imports the module and returns the class (vllm/distributed/kv_transfer/kv_connector/factory.py:36-40). The module is imported only at the first lookup.

  2. 2
    With a module path set, what must kv_connector contain?
    medium
    Show answer

    The class name inside that module: getattr(connector_module, connector_name) (vllm/distributed/kv_transfer/kv_connector/factory.py:108-114). The registry is not used.

  3. 3
    Why can create_connector raise for a hybrid model even though VllmConfig checks HMA first?
    medium
    Show answer

    VllmConfig turns the manager off only when disable_hybrid_kv_cache_manager is None (vllm/config/vllm.py:2236-2259). If the user forced the manager on, create_connector still finds HMA on and raises (vllm/distributed/kv_transfer/kv_connector/factory.py:55-60).

  4. 4
    You serve with TP = 4 and a connector registered by a general plugin. How many times does register_connector run, and how many connector objects exist?
    hard
    Show answer

    The general plugin runs once in each process: front end, engine core and 4 workers, so 6 registrations, one per process registry (vllm/plugins/__init__.py:82-90). Objects: 1 SCHEDULER in the engine core (vllm/v1/core/sched/scheduler.py:177-181) and 1 WORKER per worker (vllm/distributed/kv_transfer/kv_transfer_state.py:88-92), so 5. The front end only reads the class. (A model registry subprocess, if one runs, adds one more registration.)