envs.py: environment variables as a dict of lambdas

Code version: vLLM @ 3e44c0d · 2026-10-22

Purpose#

vllm/envs.py is the one place where vLLM declares its environment variables. Code reads a variable as envs.VLLM_X. The module computes the value from os.environ on each read, until the process turns on the environment variable cache.

Where it sits#

Panel 3 of the diagram on How configuration works shows the environment as a separate channel next to VllmConfig. Each spawned process inherits os.environ and imports this module again.

Who calls it#

• Almost every part of vLLM reads envs.X. Some modules copy a value once at import, for example from vllm.envs import VLLM_ENGINE_READY_TIMEOUT_S (vllm/v1/engine/core_client.py:29).
• EngineCore.__init__ and WorkerProc.__init__ call enable_envs_cache() at their end (vllm/v1/engine/core.py:241-243, vllm/v1/executor/multiproc_executor.py:708-710).
• cleanup_dist_env_and_memory calls disable_envs_cache() (vllm/distributed/parallel_state.py:2543-2544).
• The compile backend and the AOT cache call compile_factors() (vllm/compilation/backends.py:1059, vllm/compilation/caching.py:576).
• vllm/collect_env.py imports the dict (vllm/collect_env.py:20).

What it calls#

Only the standard library at import. The readers call os.getenv when they run. use_aot_compile imports a torch version helper (vllm/envs.py:378-387); get_vllm_port imports urllib3 only on a parse error (vllm/envs.py:532-544).

Key classes and functions#

Walk through the main methods#

The two declarations#

Each variable has a typed line in the TYPE_CHECKING block, for example:

    LOCAL_RANK: int = 0
    CUDA_VISIBLE_DEVICES: str | None = None
    VLLM_ENGINE_READY_TIMEOUT_S: int = 600
    VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT: float = 30.0

Python does not run this block, so these defaults have no effect. They give type checkers and editors a type for envs.VLLM_ENGINE_READY_TIMEOUT_S. The run-time reader is the dict entry:

    # Timeout in seconds for waiting for engine cores to become ready
    # during startup. Default is 600 seconds (10 minutes).
    "VLLM_ENGINE_READY_TIMEOUT_S": lambda: int(
        os.environ.get("VLLM_ENGINE_READY_TIMEOUT_S", "600")
    ),

The dict has 317 entries: 295 lambdas, 16 env_with_choices readers, one env_set_with_choices, one get_env_or_set_default and four named functions (derived from the source with ast). The code comment above each key is its description. Two comments split the dict: "Installation Time Env Vars" and "Runtime Env Vars" (vllm/envs.py:619, vllm/envs.py:710).

A reader with choices#

    def _get_validated_env() -> str | None:
        value = os.getenv(env_name)
        if value is None:
            return default

        # Resolve choices if it's a callable (for lazy loading)
        actual_choices = choices() if callable(choices) else choices

        if not case_sensitive:
            check_value = value.lower()
            check_choices = [choice.lower() for choice in actual_choices]
        else:
            check_value = value
            check_choices = actual_choices

        if check_value not in check_choices:
            raise ValueError(
                f"Invalid value '{value}' for {env_name}. "
                f"Valid options: {actual_choices}."
            )

        return value

    return _get_validated_env

1. Lines 420-422. Not set: the default, with no check.
2. Lines 434-438. A value that is not in the list raises ValueError. The error comes at the first read, not at start-up.

The list form splits at commas and strips spaces (vllm/envs.py:472). The plain lambda of VLLM_PLUGINS does not strip them (vllm/envs.py:1163-1167). So VLLM_PLUGINS="a, b" gives ["a", " b"], and no plugin is named " b" (derived).

__getattr__: a read is a function call#

def __getattr__(name: str):
    """Gets environment variables lazily.

    NOTE: After enable_envs_cache() invocation (which triggered after service
    initialization), all environment variables will be cached.
    """
    if name in environment_variables:
        return environment_variables[name]()
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

The module defines no attribute VLLM_X. So Python calls this module-level __getattr__ for envs.VLLM_X (PEP 562), and also for from vllm.envs import VLLM_X. Line 2298 calls the reader each time. The from ... import form keeps the value of that one moment in the importing module.

enable_envs_cache: freeze the values#

def enable_envs_cache() -> None:
    """Enables caching of environment variables. This is useful for performance
    reasons, as it avoids the need to re-evaluate environment variables on
    every call.

    NOTE: Currently, it's invoked after service initialization to reduce
    runtime overhead. This also means that environment variables should NOT
    be updated after the service is initialized.
    """
    if _is_envs_cache_enabled():
        # Avoid wrapping functools.cache multiple times
        return
    # Tag __getattr__ with functools.cache
    global __getattr__
    __getattr__ = functools.cache(__getattr__)

    # Cache all environment variables
    for key in environment_variables:
        __getattr__(key)

1. Lines 2317-2319. A second call does nothing. _is_envs_cache_enabled checks for the cache_clear attribute of functools.cache (vllm/envs.py:2302-2305).
2. Lines 2321-2322. The module __getattr__ is replaced by a cached version of itself.
3. Lines 2325-2326. Every reader runs once now, so the cache holds all 317 values.

disable_envs_cache puts the original function back through __wrapped__ (vllm/envs.py:2329-2337). is_set(name) does not use the reader: it checks name in os.environ (vllm/envs.py:2344-2348).

compile_factors: the environment in the compile cache key#

    factors: dict[str, object] = {}
    for factor, getter in environment_variables.items():
        if factor in ignored_factors:
            continue

        try:
            raw = getter()
        except Exception as exc:  # pragma: no cover - defensive logging
            logger.warning(
                "Skipping environment variable %s while hashing compile factors: %s",
                factor,
                exc,
            )
            continue

        factors[factor] = normalize_value(raw)

Every variable that is not in ignored_factors joins the key. The ignore list has 82 names, for example VLLM_PORT, VLLM_CACHE_ROOT, the logging variables and VLLM_API_KEY (vllm/envs.py:2356-2447). A reader that raises is skipped with a warning (lines 2456-2464). Nine Ray variables join the key too (lines 2468-2491). The compile backend hashes this dict into the env hash of the compile cache (vllm/compilation/backends.py:1059-1060).

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

Related concepts#

• Environment variables: every variable with its default, its reader and its users.
• How configuration works: flags, VllmConfig and the environment side by side.
• How vLLM finds and loads plugins: VLLM_PLUGINS and the cache, in practice.
• torch.compile and CUDA graphs in vLLM: the compile cache key that uses compile_factors.