vLLM Explained

v1/engine/input_processor.py: from engine input to EngineCoreRequest

IntermediateCode mapabout 8 min read + 3 min deep dives

Code version: vLLM @ 3e44c0d · 2026-10-22

Purpose#

vllm/v1/engine/input_processor.py turns one rendered prompt and its parameters into one EngineCoreRequest. On the way, it rejects requests that the engine cannot run and fills in the defaults that the engine core needs.

Filevllm/v1/engine/input_processor.py, 657 lines of Python. One class.
Main classInputProcessor (vllm/v1/engine/input_processor.py:41).
Runs inThe API server process. process_inputs runs on the event loop for rendered inputs, and in the renderer's thread pool for raw prompts (process_inputs_async).
Called byAsyncLLM.add_request and LLMEngine.add_request; the beam search code reads its fields.
CallsSamplingParams.verify, clone, update_from_generation_config, update_from_tokenizer; PoolingParams.verify; the renderer (raw prompts, token-id checks); current_platform.validate_request; the logits processor validator.
Not in this fileChat templates and tokenization (vllm/renderers/), the multimodal processor, the sending of the request (core_client.py), the engine core's Request (vllm/v1/request.py).

The name says "processor", but for vllm serve the heavy work is already done: the renderer has applied the chat template and tokenized the prompt. The input processor checks and packs.

Where it sits#

The input processor stands between the renderer and the engine core client: renderer → EngineInput dict → InputProcessor.process_inputs → EngineCoreRequest → assign_request_id → engine core client. The page Data contracts between the parts of vLLM shows the types on both sides.

Who calls it#

What it calls#

Key classes and functions#

NameLineWhat it does
InputProcessor.__init__vllm/v1/engine/input_processor.py:42-84Keeps config parts, the generation config, the renderer and the multimodal budget; makes process_inputs_async.
_build_logits_processors_params_validatorvllm/v1/engine/input_processor.py:93-113Picks the validator of Model Runner V2 or V1.
resolve_watermarking / apply_watermarkingvllm/v1/engine/input_processor.py:115-136Decide and store whether this request is watermarked.
_validate_paramsvllm/v1/engine/input_processor.py:138-252All checks of SamplingParams or PoolingParams.
_validate_loravllm/v1/engine/input_processor.py:278-295A LoRA request needs LoRA to be enabled.
assign_request_idvllm/v1/engine/input_processor.py:353-371Moves the caller's id to external_req_id and adds 8 random characters.
process_inputsvllm/v1/engine/input_processor.py:373-555The main method: checks, defaults, multimodal features, the struct.
_validate_prompt_lenvllm/v1/engine/input_processor.py:557-602Empty prompt, too long, or no room for one output token.
_validate_model_input(s)vllm/v1/engine/input_processor.py:604-657Length, embeddings mask, multimodal item size, token-id range.

Walk through the main methods#

process_inputs, part 1: checks and the engine input#

        watermarking = self._validate_params(params, supported_tasks)
        self._validate_lora(lora_request)

        parallel_config = self.vllm_config.parallel_config
        dp_size = parallel_config.data_parallel_size
        dp_local_size = parallel_config.data_parallel_size_local
        num_ranks = dp_local_size if parallel_config.local_engines_only else dp_size
        if data_parallel_rank is not None and not (0 <= data_parallel_rank < num_ranks):
            raise VLLMValidationError(
                f"data_parallel_rank {data_parallel_rank} "
                f"is out of range [0, {num_ranks})."
            )

        if isinstance(prompt, dict) and "type" in prompt:
            if arrival_time is None:
                arrival_time = prompt.get("arrival_time", time.time())  # type: ignore[assignment]

            engine_input: EngineInput = prompt  # type: ignore[assignment]
            parsed_prompt = parse_model_prompt(model_config, prompt)
            tok_params = renderer.default_cmpl_tok_params.with_kwargs(
                **(tokenization_kwargs or {})
            )

            (engine_input,) = renderer.render_cmpl(
                [parsed_prompt],
                tok_params,
            )

        current_platform.validate_request(engine_input, params)

        encoder_input, decoder_input = split_enc_dec_input(engine_input)
        self._validate_model_inputs(encoder_input, decoder_input)
  1. Lines 389-390. Check the parameters and the LoRA request first. _validate_params returns the watermark decision; the code applies it later to its own copy.
  2. Lines 392-400. An explicit data_parallel_rank must be in range. The range is the local DP size with hybrid or external load balancing, else the full DP size.
  3. Lines 402-406. The vllm serve path: the prompt is already an engine input. Its arrival_time, set by the renderer, wins over a new time.time().
  4. Lines 407-428. The old path for raw prompts logs a deprecation warning once and renders the prompt here.
  5. Line 430. The platform can reject the request.
  6. Lines 432-433. Split encoder and decoder inputs, and check their lengths and token ids.

process_inputs, part 2: sampling parameters and defaults#

        sampling_params = None
        pooling_params = None
        if isinstance(params, SamplingParams):
            # TODO: can we avoid cloning here in multiproc case?
            sampling_params = params.clone()
            # Resolve on the request-local copy: `params` may be shared across
            # prompts, requests and even engines by the caller.
            assert watermarking is not None
            self.apply_watermarking(sampling_params, watermarking)
            prompt_len = length_from_prompt_token_ids_or_embeds(
                prompt_token_ids, prompt_embeds
            )
            # If unset max tokens, then generate up to the max_model_len.
            if sampling_params.max_tokens is None:
                sampling_params.max_tokens = (
                    self.model_config.max_model_len - prompt_len
                )
                # min_tokens is not checked while max_tokens is unset.
                if sampling_params.min_tokens > sampling_params.max_tokens:
                    raise VLLMValidationError(
                        f"min_tokens must be less than or equal to "
                        f"max_tokens={sampling_params.max_tokens}, got "
                        f"{sampling_params.min_tokens}."
                    )

            sampling_params.update_from_generation_config(
                self.generation_config_fields,
                self.renderer.get_eos_token_id(),
            )
            if self.tokenizer is not None:
                sampling_params.update_from_tokenizer(self.tokenizer)
            if sampling_params.trace_decode_token_ids:
                self._normalize_trace_replay_params(sampling_params, prompt_len)
        else:
            pooling_params = params.clone()
  1. Line 449. clone(): the caller's SamplingParams object can be shared by many requests, so the code changes only a copy. The comment at lines 450-451 says so.
  2. Lines 457-475. Two range checks for rarely used fields: routed_experts_prompt_start and the rows of prompt_logprob_token_ids.
  3. Lines 476-487. No max_tokens: use all the room that is left, max_model_len − prompt_len. Only now can min_tokens be checked against it.
  4. Lines 489-492. update_from_generation_config sets the EOS token id and adds the eos_token_id values of the model's generation config to stop_token_ids. With ignore_eos, it changes neither (vllm/sampling_params.py:775-803).
  5. Lines 493-494. update_from_tokenizer turns bad_words into token ids (vllm/sampling_params.py:805-847).
  6. Line 498. Pooling parameters are cloned too, but get no defaults here.

process_inputs, part 3: the struct#

        return EngineCoreRequest(
            request_id=request_id,
            prompt_token_ids=prompt_token_ids,
            prompt_embeds=prompt_embeds,
            prompt_is_token_ids=prompt_is_token_ids,
            mm_features=mm_features,
            sampling_params=sampling_params,
            pooling_params=pooling_params,
            arrival_time=arrival_time,
            lora_request=lora_request,
            cache_salt=decoder_input.get("cache_salt"),
            priority=priority,
            data_parallel_rank=data_parallel_rank,
            trace_headers=trace_headers,
            resumable=resumable,
            session_id=session_id,
            kv_hints=kv_hints,
        )

Every field comes from the steps above. cache_salt comes from the decoder input (line 548): a client can set it to keep its prefix-cache entries apart from other clients. client_index, current_wave and external_req_id keep their defaults here. The client and assign_request_id set them later.

assign_request_id#

    @staticmethod
    def assign_request_id(request: EngineCoreRequest):
        """Replace the externally supplied request ID with an internal request ID
        that adds 8 random characters in order to ensure uniqueness.
        """
        if request.external_req_id is not None:
            raise ValueError(
                "The external_req_id field should not be set on EngineCoreRequests"
                " passed to vLLM; use the request_id field."
            )
        request.external_req_id = request.request_id
        if envs.VLLM_DISABLE_REQUEST_ID_RANDOMIZATION:
            logger.warning_once(
                "VLLM_DISABLE_REQUEST_ID_RANDOMIZATION is set and will be "
                "removed in a future release. Duplicate externally-provided "
                "request IDs may cause failures and/or subtle correctness errors."
            )
        else:
            request.request_id = f"{request.external_req_id}-{random_uuid():.8}"
  1. Lines 358-362. A caller must not set external_req_id itself.
  2. Line 363. The caller's id becomes the external id. The output processor returns it in RequestOutput.request_id.
  3. Lines 364-369. With VLLM_DISABLE_REQUEST_ID_RANDOMIZATION, the id stays unchanged, with a warning about duplicates.
  4. Line 371. Else the first 8 characters of a new UUID are appended after a dash.

_validate_params for sampling#

        if isinstance(params, SamplingParams):
            supported_generation_tasks = [
                task for task in supported_tasks if task in GENERATION_TASKS
            ]
            if not supported_generation_tasks:
                raise VLLMValidationError("This model does not support generation")

            params.verify(
                self.model_config,
                self.speculative_config,
                self.structured_outputs_config,
                self.tokenizer,
            )
            if params.prompt_logprob_token_ids is not None:
                if not self.vllm_config.use_v2_model_runner:
                    raise VLLMValidationError(
                        "prompt_logprob_token_ids requires the V2 model runner "
                        "(VLLM_USE_V2_MODEL_RUNNER=1).",
                        parameter="prompt_logprob_token_ids",
                    )
                if self.vllm_config.cache_config.kv_sharing_fast_prefill:
                    raise VLLMValidationError(
                        "prompt_logprob_token_ids is incorrect with "
                        "--kv-sharing-fast-prefill; disable it for scoring.",
                        parameter="prompt_logprob_token_ids",
                    )

            self.validate_logits_processors_params(params)
  1. Lines 151-155. A SamplingParams request needs a model with a generation task.
  2. Lines 157-162. SamplingParams.verify checks the fields against the model, the spec decoding config, the structured output config and the tokenizer.
  3. Lines 163-175. prompt_logprob_token_ids needs Model Runner V2 and does not work with --kv-sharing-fast-prefill.
  4. Line 177. Custom logits processors can check their own request arguments.

Further checks follow for diffusion models, sampling-mask replay, thinking_token_budget (needs a reasoning config) and trace replay (vllm/v1/engine/input_processor.py:179-220). Watermarking is resolved last, so a missing feature is reported before an incompatibility with watermarking (vllm/v1/engine/input_processor.py:221-223).

_validate_prompt_len#

        if prompt_len == 0 and prompt_type == "decoder":
            raise VLLMValidationError(f"The {prompt_type} prompt cannot be empty")

        model_config = self.model_config
        max_prompt_len = (
            model_config.max_model_len
            if prompt_type == "decoder"
            else self.mm_encoder_cache_size
        )
        if prompt_len > max_prompt_len:
            if self.supports_mm_inputs:
                suggestion = (
                    "Make sure that `max_model_len` is no smaller than the "
                    "number of text tokens plus multimodal tokens. For image "
                    "inputs, the number of image tokens depends on the number "
                    "of images, and possibly their aspect ratios as well."
                )
            else:
                suggestion = (
                    "Make sure that `max_model_len` is no smaller than the "
                    "number of text tokens."
                )

            raise VLLMValidationError(
                f"The {prompt_type} prompt (length {prompt_len}) is "
                f"longer than the maximum model length of {max_prompt_len}. "
                f"{suggestion}"
            )
        elif prompt_len == max_prompt_len and model_config.runner_type == "generate":
            suggestion = (
                "Make sure that `max_model_len` is no smaller than the "
                "number of text tokens (prompt + requested output tokens)."
            )
            raise VLLMValidationError(
                f"The {prompt_type} prompt (length {prompt_len}) plus the number of "
                f"requested output tokens (at least 1) is longer than the maximum "
                f"model length of {max_prompt_len}. {suggestion}"
            )
  1. Lines 565-566. An empty decoder prompt is an error.
  2. Lines 569-573. The limit is max_model_len for the decoder and the encoder cache size for an encoder prompt.
  3. Lines 574-591. Longer than the limit: error, with a hint for multimodal models.
  4. Lines 593-602. Exactly the limit, for a generative model: error too, because no room is left for one output token.

Deep dive#

The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.

Deep dive

Data flow: types in and out

ValueTypeWhere
promptEngineInput (a TypedDict such as TokensInput: type, prompt_token_ids, optional prompt, arrival_time, cache_salt) or a raw PromptTypevllm/v1/engine/input_processor.py:376, vllm/inputs/engine.py:18-40
paramsSamplingParams | PoolingParamsvllm/v1/engine/input_processor.py:377
supported_taskstuple[SupportedTask, ...], from the engine corevllm/v1/engine/input_processor.py:378
multimodal featureslist[MultiModalFeatureSpec], sorted by position in the promptvllm/v1/engine/input_processor.py:500-536
resultEngineCoreRequestvllm/v1/engine/input_processor.py:538-555
Deep dive

Processes and threads

The class lives in the API server process. It has no lock and no thread of its own. process_inputs_async is make_async(process_inputs) with the renderer's executor (vllm/v1/engine/input_processor.py:79-84). So raw-prompt requests run in the same thread pool as the tokenizer. The comment explains why: tokenization and multimodal processing block, and must not block the event loop.

The engine core repeats a few checks. For example, it checks the pooling task again in EngineCore.add_request (vllm/v1/engine/core.py:520-529). Structured-output grammars are compiled only in the engine core. The input thread starts the compilation in a thread pool of the structured output manager (vllm/v1/engine/core.py:1129-1135, vllm/v1/structured_output/__init__.py:196-197).

Deep dive

Worked example: three prompts and max_model_len = 32768

Assume a generative model with max_model_len = 32768 and a request with no max_tokens and no min_tokens.

Prompt lengthResultWhy
61max_tokens = 3270732768 − 61 (lines 477-480)
32768VLLMValidationError: "plus the number of requested output tokens (at least 1) is longer"equal to the limit, generative model (lines 593-602)
40000VLLMValidationError: "longer than the maximum model length of 32768"line 574

With vllm serve, the renderer usually rejects a too-long prompt first, during tokenization (vllm/renderers/params.py:518-538). The checks here also protect the paths that skip the renderer.

The EOS ids work like this (illustrative ids). The tokenizer's EOS id is 151645, and the generation config lists eos_token_id: [151645, 151643]. Then _eos_token_id = 151645, and 151643 is added to stop_token_ids, because the code removes the primary EOS id from the set (vllm/sampling_params.py:789-803).

Deep dive

Error paths: what raises, what only logs

SituationResultWhere
sampling request for a pooling-only modelVLLMValidationError("This model does not support generation")vllm/v1/engine/input_processor.py:151-155
pooling task not supportedVLLMValidationError with the supported tasksvllm/v1/engine/input_processor.py:239-243
params of another typeTypeErrorvllm/v1/engine/input_processor.py:246-250
LoRA request without --enable-loraVLLMValidationErrorvllm/v1/engine/input_processor.py:283-286
data_parallel_rank out of rangeVLLMValidationErrorvllm/v1/engine/input_processor.py:396-400
raw prompt instead of a rendered inputdeprecation warning, oncevllm/v1/engine/input_processor.py:408-412
min_tokens larger than the computed max_tokensVLLMValidationErrorvllm/v1/engine/input_processor.py:481-487
multimodal hashes that are not stringsValueError (a broken custom processor)vllm/v1/engine/input_processor.py:508-515
multimodal item larger than the encoder cacheVLLMValidationError that names --limit-mm-per-promptvllm/v1/engine/input_processor.py:630-643
caller set external_req_idValueErrorvllm/v1/engine/input_processor.py:358-362

VLLMValidationError becomes HTTP 400 in the API server (vllm/entrypoints/serve/exception_handling/error_response.py:43-46).

Deep dive

Environment variables

VariableDefaultRead atEffect
VLLM_DISABLE_REQUEST_ID_RANDOMIZATION0 (vllm/envs.py:1715-1716)vllm/v1/engine/input_processor.py:364Keeps the caller's request id; deprecated.
VLLM_MAX_NUM_BAD_WORDS128 (vllm/envs.py:1152-1154)vllm/sampling_params.py:809Limit for the tokenized bad_words; more raises VLLMValidationError.

The full list of variables is on Environment variables.

Check yourself#

  1. 1
    A chat request has no max_tokens. The prompt has 1,000 tokens, and max_model_len is 8192. Which max_tokens reaches the engine core?
    easy
    Show answer

    7192, if the server and the platform set no smaller limit. The chat serving layer computes it with get_max_tokens, the smallest of max_model_len − prompt_len and the other limits (vllm/entrypoints/serve/utils/api_utils.py:253-270). process_inputs would fill the same value if it were still None (vllm/v1/engine/input_processor.py:476-480).

  2. 2
    Why does process_inputs clone the SamplingParams?
    medium
    Show answer

    The caller's object can be shared across prompts, requests and engines. The code then changes the copy: watermarking, max_tokens, EOS and stop ids (vllm/v1/engine/input_processor.py:447-453, vllm/v1/engine/input_processor.py:476-494).

  3. 3
    The client sends the id abc. Which ids do the output processor and the engine core use?
    medium
    Show answer

    The engine core uses abc- plus 8 random characters, for example abc-1f2e3d4c. The output processor keeps abc as external_req_id and puts it into RequestOutput.request_id (vllm/v1/engine/input_processor.py:363-371). (For chat, the serving layer has already added the prefix chatcmpl-.)

  4. 4
    A model has max_model_len = 4096. A prompt of exactly 4096 tokens reaches process_inputs. What happens, and would an embedding model behave the same?
    hard
    Show answer

    The generative model rejects it: the prompt length equals the limit, and the runner type is "generate", so no room is left for one output token (vllm/v1/engine/input_processor.py:593-602). An embedding model has another runner type, so a prompt of exactly 4096 tokens passes; only a longer one fails (vllm/v1/engine/input_processor.py:574).