v1/engine/input_processor.py: from engine input to EngineCoreRequest
Code version: vLLM @ 3e44c0d · 2026-10-22
Purpose#
vllm/v1/engine/input_processor.py turns one rendered prompt and its parameters into one EngineCoreRequest. On the way, it rejects requests that the engine cannot run and fills in the defaults that the engine core needs.
| File | vllm/v1/engine/input_processor.py, 657 lines of Python. One class. |
|---|---|
| Main class | InputProcessor (vllm/v1/engine/input_processor.py:41). |
| Runs in | The API server process. process_inputs runs on the event loop for rendered inputs, and in the renderer's thread pool for raw prompts (process_inputs_async). |
| Called by | AsyncLLM.add_request and LLMEngine.add_request; the beam search code reads its fields. |
| Calls | SamplingParams.verify, clone, update_from_generation_config, update_from_tokenizer; PoolingParams.verify; the renderer (raw prompts, token-id checks); current_platform.validate_request; the logits processor validator. |
| Not in this file | Chat templates and tokenization (vllm/renderers/), the multimodal processor, the sending of the request (core_client.py), the engine core's Request (vllm/v1/request.py). |
The name says "processor", but for vllm serve the heavy work is already done: the renderer has applied the chat template and tokenized the prompt. The input processor checks and packs.
Where it sits#
The input processor stands between the renderer and the engine core client: renderer → EngineInput dict → InputProcessor.process_inputs → EngineCoreRequest → assign_request_id → engine core client. The page Data contracts between the parts of vLLM shows the types on both sides.
Who calls it#
AsyncLLM.__init__makes one instance and gives it the shared renderer (vllm/v1/engine/async_llm.py:170).LLMEnginedoes the same (vllm/v1/engine/llm_engine.py:101).AsyncLLM.add_requestcallsprocess_inputsfor rendered inputs andprocess_inputs_asyncfor raw prompts, thenassign_request_id(vllm/v1/engine/async_llm.py:481-521).- The serving classes keep a reference to it as
self.input_processor(vllm/entrypoints/generate/base/serving.py:151). Online beam search callsresolve_watermarkingand readsgeneration_config_fields(vllm/entrypoints/generate/beam_search/online.py:49,vllm/entrypoints/generate/beam_search/online.py:78).
What it calls#
SamplingParams.verifyandPoolingParams.verifyfor the parameter rules (vllm/v1/engine/input_processor.py:157-162,vllm/v1/engine/input_processor.py:245).- The logits processor validator, built once in the constructor. With Model Runner V2 it loads the custom logits processor classes (
vllm/v1/engine/input_processor.py:93-113). current_platform.validate_request, a hook for platform plugins (vllm/v1/engine/input_processor.py:430).renderer.validate_token_ids, which rejects token ids outside the vocabulary (vllm/v1/engine/input_processor.py:646-647,vllm/renderers/base.py:453-458).
Key classes and functions#
| Name | Line | What it does |
|---|---|---|
InputProcessor.__init__ | vllm/v1/engine/input_processor.py:42-84 | Keeps config parts, the generation config, the renderer and the multimodal budget; makes process_inputs_async. |
_build_logits_processors_params_validator | vllm/v1/engine/input_processor.py:93-113 | Picks the validator of Model Runner V2 or V1. |
resolve_watermarking / apply_watermarking | vllm/v1/engine/input_processor.py:115-136 | Decide and store whether this request is watermarked. |
_validate_params | vllm/v1/engine/input_processor.py:138-252 | All checks of SamplingParams or PoolingParams. |
_validate_lora | vllm/v1/engine/input_processor.py:278-295 | A LoRA request needs LoRA to be enabled. |
assign_request_id | vllm/v1/engine/input_processor.py:353-371 | Moves the caller's id to external_req_id and adds 8 random characters. |
process_inputs | vllm/v1/engine/input_processor.py:373-555 | The main method: checks, defaults, multimodal features, the struct. |
_validate_prompt_len | vllm/v1/engine/input_processor.py:557-602 | Empty prompt, too long, or no room for one output token. |
_validate_model_input(s) | vllm/v1/engine/input_processor.py:604-657 | Length, embeddings mask, multimodal item size, token-id range. |
Walk through the main methods#
process_inputs, part 1: checks and the engine input#
watermarking = self._validate_params(params, supported_tasks)
self._validate_lora(lora_request)
parallel_config = self.vllm_config.parallel_config
dp_size = parallel_config.data_parallel_size
dp_local_size = parallel_config.data_parallel_size_local
num_ranks = dp_local_size if parallel_config.local_engines_only else dp_size
if data_parallel_rank is not None and not (0 <= data_parallel_rank < num_ranks):
raise VLLMValidationError(
f"data_parallel_rank {data_parallel_rank} "
f"is out of range [0, {num_ranks})."
)
if isinstance(prompt, dict) and "type" in prompt:
if arrival_time is None:
arrival_time = prompt.get("arrival_time", time.time()) # type: ignore[assignment]
engine_input: EngineInput = prompt # type: ignore[assignment] parsed_prompt = parse_model_prompt(model_config, prompt)
tok_params = renderer.default_cmpl_tok_params.with_kwargs(
**(tokenization_kwargs or {})
)
(engine_input,) = renderer.render_cmpl(
[parsed_prompt],
tok_params,
)
current_platform.validate_request(engine_input, params)
encoder_input, decoder_input = split_enc_dec_input(engine_input)
self._validate_model_inputs(encoder_input, decoder_input)- Lines 389-390. Check the parameters and the LoRA request first.
_validate_paramsreturns the watermark decision; the code applies it later to its own copy. - Lines 392-400. An explicit
data_parallel_rankmust be in range. The range is the local DP size with hybrid or external load balancing, else the full DP size. - Lines 402-406. The
vllm servepath: the prompt is already an engine input. Itsarrival_time, set by the renderer, wins over a newtime.time(). - Lines 407-428. The old path for raw prompts logs a deprecation warning once and renders the prompt here.
- Line 430. The platform can reject the request.
- Lines 432-433. Split encoder and decoder inputs, and check their lengths and token ids.
process_inputs, part 2: sampling parameters and defaults#
sampling_params = None
pooling_params = None
if isinstance(params, SamplingParams):
# TODO: can we avoid cloning here in multiproc case?
sampling_params = params.clone()
# Resolve on the request-local copy: `params` may be shared across
# prompts, requests and even engines by the caller.
assert watermarking is not None
self.apply_watermarking(sampling_params, watermarking)
prompt_len = length_from_prompt_token_ids_or_embeds(
prompt_token_ids, prompt_embeds
) # If unset max tokens, then generate up to the max_model_len.
if sampling_params.max_tokens is None:
sampling_params.max_tokens = (
self.model_config.max_model_len - prompt_len
)
# min_tokens is not checked while max_tokens is unset.
if sampling_params.min_tokens > sampling_params.max_tokens:
raise VLLMValidationError(
f"min_tokens must be less than or equal to "
f"max_tokens={sampling_params.max_tokens}, got "
f"{sampling_params.min_tokens}."
)
sampling_params.update_from_generation_config(
self.generation_config_fields,
self.renderer.get_eos_token_id(),
)
if self.tokenizer is not None:
sampling_params.update_from_tokenizer(self.tokenizer)
if sampling_params.trace_decode_token_ids:
self._normalize_trace_replay_params(sampling_params, prompt_len)
else:
pooling_params = params.clone()
- Line 449.
clone(): the caller'sSamplingParamsobject can be shared by many requests, so the code changes only a copy. The comment at lines 450-451 says so. - Lines 457-475. Two range checks for rarely used fields:
routed_experts_prompt_startand the rows ofprompt_logprob_token_ids. - Lines 476-487. No
max_tokens: use all the room that is left,max_model_len − prompt_len. Only now canmin_tokensbe checked against it. - Lines 489-492.
update_from_generation_configsets the EOS token id and adds theeos_token_idvalues of the model's generation config tostop_token_ids. Withignore_eos, it changes neither (vllm/sampling_params.py:775-803). - Lines 493-494.
update_from_tokenizerturnsbad_wordsinto token ids (vllm/sampling_params.py:805-847). - Line 498. Pooling parameters are cloned too, but get no defaults here.
process_inputs, part 3: the struct#
return EngineCoreRequest(
request_id=request_id,
prompt_token_ids=prompt_token_ids,
prompt_embeds=prompt_embeds,
prompt_is_token_ids=prompt_is_token_ids,
mm_features=mm_features,
sampling_params=sampling_params,
pooling_params=pooling_params,
arrival_time=arrival_time,
lora_request=lora_request,
cache_salt=decoder_input.get("cache_salt"),
priority=priority,
data_parallel_rank=data_parallel_rank,
trace_headers=trace_headers,
resumable=resumable,
session_id=session_id,
kv_hints=kv_hints,
)Every field comes from the steps above. cache_salt comes from the decoder input (line 548): a client can set it to keep its prefix-cache entries apart from other clients. client_index, current_wave and external_req_id keep their defaults here. The client and assign_request_id set them later.
assign_request_id#
@staticmethod
def assign_request_id(request: EngineCoreRequest):
"""Replace the externally supplied request ID with an internal request ID
that adds 8 random characters in order to ensure uniqueness.
"""
if request.external_req_id is not None:
raise ValueError(
"The external_req_id field should not be set on EngineCoreRequests"
" passed to vLLM; use the request_id field."
)
request.external_req_id = request.request_id
if envs.VLLM_DISABLE_REQUEST_ID_RANDOMIZATION:
logger.warning_once(
"VLLM_DISABLE_REQUEST_ID_RANDOMIZATION is set and will be "
"removed in a future release. Duplicate externally-provided "
"request IDs may cause failures and/or subtle correctness errors."
)
else:
request.request_id = f"{request.external_req_id}-{random_uuid():.8}"- Lines 358-362. A caller must not set
external_req_iditself. - Line 363. The caller's id becomes the external id. The output processor returns it in
RequestOutput.request_id. - Lines 364-369. With
VLLM_DISABLE_REQUEST_ID_RANDOMIZATION, the id stays unchanged, with a warning about duplicates. - Line 371. Else the first 8 characters of a new UUID are appended after a dash.
_validate_params for sampling#
if isinstance(params, SamplingParams):
supported_generation_tasks = [
task for task in supported_tasks if task in GENERATION_TASKS
]
if not supported_generation_tasks:
raise VLLMValidationError("This model does not support generation")
params.verify(
self.model_config,
self.speculative_config,
self.structured_outputs_config,
self.tokenizer,
)
if params.prompt_logprob_token_ids is not None:
if not self.vllm_config.use_v2_model_runner:
raise VLLMValidationError(
"prompt_logprob_token_ids requires the V2 model runner "
"(VLLM_USE_V2_MODEL_RUNNER=1).",
parameter="prompt_logprob_token_ids",
)
if self.vllm_config.cache_config.kv_sharing_fast_prefill:
raise VLLMValidationError(
"prompt_logprob_token_ids is incorrect with "
"--kv-sharing-fast-prefill; disable it for scoring.",
parameter="prompt_logprob_token_ids",
)
self.validate_logits_processors_params(params)- Lines 151-155. A
SamplingParamsrequest needs a model with a generation task. - Lines 157-162.
SamplingParams.verifychecks the fields against the model, the spec decoding config, the structured output config and the tokenizer. - Lines 163-175.
prompt_logprob_token_idsneeds Model Runner V2 and does not work with--kv-sharing-fast-prefill. - Line 177. Custom logits processors can check their own request arguments.
Further checks follow for diffusion models, sampling-mask replay, thinking_token_budget (needs a reasoning config) and trace replay (vllm/v1/engine/input_processor.py:179-220). Watermarking is resolved last, so a missing feature is reported before an incompatibility with watermarking (vllm/v1/engine/input_processor.py:221-223).
_validate_prompt_len#
if prompt_len == 0 and prompt_type == "decoder":
raise VLLMValidationError(f"The {prompt_type} prompt cannot be empty")
model_config = self.model_config
max_prompt_len = (
model_config.max_model_len
if prompt_type == "decoder"
else self.mm_encoder_cache_size
)
if prompt_len > max_prompt_len:
if self.supports_mm_inputs:
suggestion = (
"Make sure that `max_model_len` is no smaller than the "
"number of text tokens plus multimodal tokens. For image "
"inputs, the number of image tokens depends on the number "
"of images, and possibly their aspect ratios as well."
)
else:
suggestion = (
"Make sure that `max_model_len` is no smaller than the "
"number of text tokens."
)
raise VLLMValidationError(
f"The {prompt_type} prompt (length {prompt_len}) is "
f"longer than the maximum model length of {max_prompt_len}. "
f"{suggestion}"
)
elif prompt_len == max_prompt_len and model_config.runner_type == "generate":
suggestion = (
"Make sure that `max_model_len` is no smaller than the "
"number of text tokens (prompt + requested output tokens)."
)
raise VLLMValidationError(
f"The {prompt_type} prompt (length {prompt_len}) plus the number of "
f"requested output tokens (at least 1) is longer than the maximum "
f"model length of {max_prompt_len}. {suggestion}"
)- Lines 565-566. An empty decoder prompt is an error.
- Lines 569-573. The limit is
max_model_lenfor the decoder and the encoder cache size for an encoder prompt. - Lines 574-591. Longer than the limit: error, with a hint for multimodal models.
- Lines 593-602. Exactly the limit, for a generative model: error too, because no room is left for one output token.
Deep dive#
The sections below give the full detail. Open them one at a time, or use Open all deep dives at the top of the page.
Deep diveData flow: types in and out
| Value | Type | Where |
|---|---|---|
prompt | EngineInput (a TypedDict such as TokensInput: type, prompt_token_ids, optional prompt, arrival_time, cache_salt) or a raw PromptType | vllm/v1/engine/input_processor.py:376, vllm/inputs/engine.py:18-40 |
params | SamplingParams | PoolingParams | vllm/v1/engine/input_processor.py:377 |
supported_tasks | tuple[SupportedTask, ...], from the engine core | vllm/v1/engine/input_processor.py:378 |
| multimodal features | list[MultiModalFeatureSpec], sorted by position in the prompt | vllm/v1/engine/input_processor.py:500-536 |
| result | EngineCoreRequest | vllm/v1/engine/input_processor.py:538-555 |
Deep diveProcesses and threads
The class lives in the API server process. It has no lock and no thread of its own. process_inputs_async is make_async(process_inputs) with the renderer's executor (vllm/v1/engine/input_processor.py:79-84). So raw-prompt requests run in the same thread pool as the tokenizer. The comment explains why: tokenization and multimodal processing block, and must not block the event loop.
The engine core repeats a few checks. For example, it checks the pooling task again in EngineCore.add_request (vllm/v1/engine/core.py:520-529). Structured-output grammars are compiled only in the engine core. The input thread starts the compilation in a thread pool of the structured output manager (vllm/v1/engine/core.py:1129-1135, vllm/v1/structured_output/__init__.py:196-197).
Deep diveWorked example: three prompts and max_model_len = 32768
Assume a generative model with max_model_len = 32768 and a request with no max_tokens and no min_tokens.
| Prompt length | Result | Why |
|---|---|---|
| 61 | max_tokens = 32707 | 32768 − 61 (lines 477-480) |
| 32768 | VLLMValidationError: "plus the number of requested output tokens (at least 1) is longer" | equal to the limit, generative model (lines 593-602) |
| 40000 | VLLMValidationError: "longer than the maximum model length of 32768" | line 574 |
With vllm serve, the renderer usually rejects a too-long prompt first, during tokenization (vllm/renderers/params.py:518-538). The checks here also protect the paths that skip the renderer.
The EOS ids work like this (illustrative ids). The tokenizer's EOS id is 151645, and the generation config lists eos_token_id: [151645, 151643]. Then _eos_token_id = 151645, and 151643 is added to stop_token_ids, because the code removes the primary EOS id from the set (vllm/sampling_params.py:789-803).
Deep diveError paths: what raises, what only logs
| Situation | Result | Where |
|---|---|---|
| sampling request for a pooling-only model | VLLMValidationError("This model does not support generation") | vllm/v1/engine/input_processor.py:151-155 |
| pooling task not supported | VLLMValidationError with the supported tasks | vllm/v1/engine/input_processor.py:239-243 |
| params of another type | TypeError | vllm/v1/engine/input_processor.py:246-250 |
LoRA request without --enable-lora | VLLMValidationError | vllm/v1/engine/input_processor.py:283-286 |
data_parallel_rank out of range | VLLMValidationError | vllm/v1/engine/input_processor.py:396-400 |
| raw prompt instead of a rendered input | deprecation warning, once | vllm/v1/engine/input_processor.py:408-412 |
min_tokens larger than the computed max_tokens | VLLMValidationError | vllm/v1/engine/input_processor.py:481-487 |
| multimodal hashes that are not strings | ValueError (a broken custom processor) | vllm/v1/engine/input_processor.py:508-515 |
| multimodal item larger than the encoder cache | VLLMValidationError that names --limit-mm-per-prompt | vllm/v1/engine/input_processor.py:630-643 |
caller set external_req_id | ValueError | vllm/v1/engine/input_processor.py:358-362 |
VLLMValidationError becomes HTTP 400 in the API server (vllm/entrypoints/serve/exception_handling/error_response.py:43-46).
Deep diveEnvironment variables
| Variable | Default | Read at | Effect |
|---|---|---|---|
VLLM_DISABLE_REQUEST_ID_RANDOMIZATION | 0 (vllm/envs.py:1715-1716) | vllm/v1/engine/input_processor.py:364 | Keeps the caller's request id; deprecated. |
VLLM_MAX_NUM_BAD_WORDS | 128 (vllm/envs.py:1152-1154) | vllm/sampling_params.py:809 | Limit for the tokenized bad_words; more raises VLLMValidationError. |
The full list of variables is on Environment variables.
Related concepts#
- The engine client: AsyncLLM in the API server: where
add_requestcalls this class. - Data contracts between the parts of vLLM: the fields of
EngineCoreRequest. - Renderers and chat templates: from messages to token ids: the step before, which makes the engine input.
- Sampling and logits processors in Model Runner V2: the parameters that this file checks.
- Tour 1, stop 5: this file in the path of one request.
Check yourself#
- 1A chat request has noeasy
max_tokens. The prompt has 1,000 tokens, andmax_model_lenis 8192. Whichmax_tokensreaches the engine core?Show answer
7192, if the server and the platform set no smaller limit. The chat serving layer computes it with
get_max_tokens, the smallest ofmax_model_len − prompt_lenand the other limits (vllm/entrypoints/serve/utils/api_utils.py:253-270).process_inputswould fill the same value if it were stillNone(vllm/v1/engine/input_processor.py:476-480). - 2Why doesmedium
process_inputsclone theSamplingParams?Show answer
The caller's object can be shared across prompts, requests and engines. The code then changes the copy: watermarking,
max_tokens, EOS and stop ids (vllm/v1/engine/input_processor.py:447-453,vllm/v1/engine/input_processor.py:476-494). - 3The client sends the idmedium
abc. Which ids do the output processor and the engine core use?Show answer
The engine core uses
abc-plus 8 random characters, for exampleabc-1f2e3d4c. The output processor keepsabcasexternal_req_idand puts it intoRequestOutput.request_id(vllm/v1/engine/input_processor.py:363-371). (For chat, the serving layer has already added the prefixchatcmpl-.) - 4A model hashard
max_model_len = 4096. A prompt of exactly 4096 tokens reachesprocess_inputs. What happens, and would an embedding model behave the same?Show answer
The generative model rejects it: the prompt length equals the limit, and the runner type is
"generate", so no room is left for one output token (vllm/v1/engine/input_processor.py:593-602). An embedding model has another runner type, so a prompt of exactly 4096 tokens passes; only a longer one fails (vllm/v1/engine/input_processor.py:574).