Environment variables

Code version: vLLM @ 3e44c0d · 2026-10-22

Use this page in two ways. To learn the mechanism, read the first three sections. To look up one variable, go to All environment variables, or search for its name. The script tools/gen_env_vars.py makes the big tables from vllm/envs.py. The text above the tables is written by hand from the code.

How vLLM declares an environment variable#

An environment variable is a named text value that a process gets from its parent when it starts. vLLM declares each of its variables two times in vllm/envs.py.

1. For type checkers. The if TYPE_CHECKING: block at the top of the file gives each variable a type and a default. Python does not run this block, so these defaults have no effect at run time:

   vllm/envs.py:14-20GitHub ↗Copy

   14151617181920

   if TYPE_CHECKING:
       VLLM_HOST_IP: str = ""
       VLLM_PORT: int | None = None
       VLLM_RPC_BASE_PATH: str = tempfile.gettempdir()
       VLLM_USE_MODELSCOPE: bool = False
       VLLM_USE_FASTOKENS: bool = False
       VLLM_RINGBUFFER_WARNING_INTERVAL: int = 60

2. For the run time. The dict environment_variables maps each name to a function with no arguments, mostly a lambda (vllm/envs.py:618). The function reads os.environ, parses the text and returns the value. Its fallback value is the real default:

   vllm/envs.py:858-878GitHub ↗Copy

   858859860861862863864865866867868869870871872873874875876877878

       "VLLM_LOGGING_CONFIG_PATH": lambda: os.getenv("VLLM_LOGGING_CONFIG_PATH"),
       # this is used for configuring the default logging level
       "VLLM_LOGGING_LEVEL": lambda: os.getenv("VLLM_LOGGING_LEVEL", "INFO").upper(),
       # this is used for configuring the default logging stream
       "VLLM_LOGGING_STREAM": lambda: os.getenv("VLLM_LOGGING_STREAM", "ext://sys.stdout"),
       # if set, VLLM_LOGGING_PREFIX will be prepended to all log messages
       "VLLM_LOGGING_PREFIX": lambda: os.getenv("VLLM_LOGGING_PREFIX", ""),
       # Controls colored logging output. Options: "auto" (default, colors when terminal),
       # "1" (always use colors), "0" (never use colors)
       "VLLM_LOGGING_COLOR": lambda: os.getenv("VLLM_LOGGING_COLOR", "auto"),
       # Standard unix flag for disabling ANSI color codes
       "NO_COLOR": lambda: os.getenv("NO_COLOR", "0") != "0",
       # De-facto standard flag for forcing ANSI color codes (e.g. non-tty case)
       "FORCE_COLOR": lambda: os.getenv("FORCE_COLOR", "0") != "0",
       # If set, vllm will log stats at this interval in seconds
       # If not set, vllm will log stats every 10 seconds.
       "VLLM_LOG_STATS_INTERVAL": lambda: (
           val
           if (val := float(os.getenv("VLLM_LOG_STATS_INTERVAL", "10."))) > 0.0
           else 10.0
       ),

Other code imports the module (import vllm.envs as envs) and reads envs.VLLM_LOGGING_LEVEL. The module has no such attribute. So Python calls the module-level __getattr__, which calls the lambda:

def __getattr__(name: str):
    """Gets environment variables lazily.

    NOTE: After enable_envs_cache() invocation (which triggered after service
    initialization), all environment variables will be cached.
    """
    if name in environment_variables:
        return environment_variables[name]()
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

Two more helpers use the dict. envs.is_set("NAME") tells if the variable is in os.environ at all (vllm/envs.py:2344-2348). __dir__ returns all the names (vllm/envs.py:2340-2341).

How the text becomes a value#

Each lambda parses its own text. There is no shared rule for a boolean, so the same text can give different results. These are the common forms:

Variables with a fixed list of values use env_with_choices. It returns the default when the variable is not set, and raises ValueError for a value that is not in the list (vllm/envs.py:419-440). VLLM_WORKER_MULTIPROC_METHOD is an example (vllm/envs.py:938-940).

When vLLM reads a variable#

A read through envs.X runs the lambda again each time. So a change to os.environ has an effect at the next read. This stays true until the process calls enable_envs_cache():

def enable_envs_cache() -> None:
    """Enables caching of environment variables. This is useful for performance
    reasons, as it avoids the need to re-evaluate environment variables on
    every call.

    NOTE: Currently, it's invoked after service initialization to reduce
    runtime overhead. This also means that environment variables should NOT
    be updated after the service is initialized.
    """
    if _is_envs_cache_enabled():
        # Avoid wrapping functools.cache multiple times
        return
    # Tag __getattr__ with functools.cache
    global __getattr__
    __getattr__ = functools.cache(__getattr__)

    # Cache all environment variables
    for key in environment_variables:
        __getattr__(key)

The function wraps __getattr__ in functools.cache and then reads every variable one time. After that, each read returns the cached value. Two places call it, at the end of their start-up:

• the EngineCore, as the last statement of EngineCore.__init__, after the executor and the KV cache are ready (vllm/v1/engine/core.py:241-243, vllm/v1/engine/core.py:147, vllm/v1/engine/core.py:160). This also covers a worker that runs inside the engine core process;
• each worker process of the multiprocessing executor, at the end of WorkerProc.__init__ (vllm/v1/executor/multiproc_executor.py:708-710).

No code in vllm/entrypoints/ calls it. So the front end process of vllm serve reads the environment again at each access. A code comment in the metrics code says the same: "processes that never enable the envs cache (the API server)" (vllm/v1/metrics/stats.py:479-481). cleanup_dist_env_and_memory removes the cache again with disable_envs_cache() (vllm/distributed/parallel_state.py:2543-2544).

Some code copies a value one time, at import, and the lambda never runs again for that copy:

• from vllm.envs import VLLM_ENGINE_READY_TIMEOUT_S in the engine core client (vllm/v1/engine/core_client.py:29). The start-up wait uses this copy (vllm/v1/engine/core_client.py:757-759).
• The class body of the front-end arguments: log_error_stack: bool = envs.VLLM_SERVER_DEV_MODE (vllm/entrypoints/common/cli_args.py:201).
• The default logging dict of vllm/logger.py, a module-level value (vllm/logger.py:75, vllm/logger.py:82).

Child processes get a copy of the parent environment when they start. The vllm CLI sets VLLM_WORKER_MULTIPROC_METHOD=spawn when the variable is not set (vllm/entrypoints/common/startup.py:147-149). The engine core processes and the worker processes get their start method from get_mp_context() (vllm/v1/engine/utils.py:256, vllm/v1/executor/multiproc_executor.py:723). With spawn, they start as new Python interpreters. They import vllm.envs again and read the environment that they got at start.

How to read the tables#

• Groups. envs.py has only two section comments: "Installation Time Env Vars" and "Runtime Env Vars" (vllm/envs.py:619, vllm/envs.py:710). The group "Installation time" keeps the code's placement. The script splits the runtime variables into topics by name patterns. The topics are the tool's choice, not a structure in the code.
• Type and default come from the TYPE_CHECKING block. When the lambda falls back to another value, the row says so (but the read falls back to). A few variables have no TYPE_CHECKING line; the row then shows the fallback of the lambda.
• Description is the code comment above the dict key, quoted as it is. It is not in Simplified Technical English.
• Used in vllm/ counts the lines that read envs.NAME, from vllm.envs import NAME or envs.is_set("NAME"), and links the first five. "No envs.* use" means that no Python file in vllm/ reads the variable through envs. The row then names other files that mention it, for example setup.py or C++ code.

The 15 variables a serving operator meets first#

This list is a choice for online serving with vllm serve. Each default is the fallback of the lambda, not the TYPE_CHECKING value.

Known issues#

Three defaults in TYPE_CHECKING do not match the read#

For these variables, the type hint says one default, but the lambda falls back to another. The lambda value is the real one.

A declared variable that no code reads#

VLLM_XLA_CHECK_RECOMPILATION is in the dict (vllm/envs.py:1086-1088), but no file in the repository mentions the name outside envs.py. So the variable has no effect.

Variables read outside envs.py#

Some names are in the dict, but code outside vllm/, or code that reads os.environ, uses them:

• C++: VLLM_ROCM_SLEEP_MEM_CHUNK_SIZE is read with getenv in csrc/cumem_allocator.cpp (csrc/cumem_allocator.cpp:26). The lambda and the envs cache do not apply.
• Rust: VLLM_CHAT_TEMPLATE_FUEL is read by the experimental Rust front end (rust/src/chat/src/renderer/hf/mod.rs:106). No Python file outside envs.py reads it, so the Python front end does not use it.
• Python, with os.environ: VLLM_ENABLE_CUDA_COMPATIBILITY and VLLM_CUDA_COMPATIBILITY_PATH (vllm/env_override.py:62-69), and VLLM_TRITON_FORCE_FIRST_CONFIG (vllm/env_override.py:976). These reads skip the lambda and the envs cache.
• Build time only: MAX_JOBS, NVCC_THREADS, VLLM_USE_PRECOMPILED and other variables of the "Installation time" group have no reader in vllm/. setup.py loads vllm/envs.py as a separate module and reads them through it (setup.py:56, setup.py:218), so for these variables the lambda still gives the value. The Docker files also set them. The generated table names those files.

All environment variables#

317 variables in environment_variables (envs.py:618), in 20 groups. 24 have no envs.* use in vllm/. Type and default come from the TYPE_CHECKING block (envs.py:14).

Installation time (20)#

ROCm (AMD) (32)#

CPU backend (7)#

Intel XPU (4)#

TPU / XLA (4)#

Logging and usage statistics (19)#

Debugging and profiling (10)#

torch.compile and CUDA graphs (13)#

KV transfer, offload and disaggregation (23)#

MoE and expert parallelism (28)#

CUDA and library paths (8)#

Engine core and model runner (10)#

Distributed and parallel execution (35)#

Multimodal and media (22)#

LoRA (5)#

Structured output and tool calling (9)#

API server and frontend (22)#

Model loading, storage and caches (13)#

Kernels, attention and quantization (32)#

Build and packaging (1)#