The OpenAI-compatible API

build_app() makes the FastAPI app and calls register_api_routers() (vllm/entrypoints/launchers/app.py:19-45). That function adds the routers in groups. Some groups depend on the tasks of the model or on a flag (vllm/entrypoints/launchers/api_server/routers.py:12-90). After that, endpoint plugins can add their own routes (vllm/entrypoints/launchers/app.py:47-51).

The health check. GET /health asks the engine client. It returns 200 when the engine is alive and 503 after an EngineDeadError (vllm/entrypoints/serve/instrumentator/health.py:22-33).

API keys. With --api-key (or VLLM_API_KEY when the flag is not set), a middleware checks the Authorization: Bearer header (vllm/entrypoints/serve/middleware/register.py:32-36). It guards only paths that start with /v1, /v2, /inference or /cohere, and answers 401 otherwise (vllm/entrypoints/serve/middleware/authenticate.py:11, vllm/entrypoints/serve/middleware/authenticate.py:59-60). So /health and /metrics stay open.

The table lists the OpenAI fields of ChatCompletionRequest that change how the server works (vllm/entrypoints/openai/chat_completion/protocol.py:228-249).

vLLM's extra fields. These are not in the OpenAI specification. The OpenAI SDK sends them in its extra_body argument.

• Sampling: top_k, min_p, repetition_penalty, min_tokens, stop_token_ids, ignore_eos, use_beam_search and others (vllm/entrypoints/openai/chat_completion/protocol.py:278-288). The basics of these are on Sampling↗.
• Chat template: chat_template, chat_template_kwargs, add_generation_prompt, continue_final_message (vllm/entrypoints/openai/chat_completion/protocol.py:324-377). For example, "chat_template_kwargs": {"enable_thinking": false} reaches a Qwen3 template.
• Scheduling: priority (lower is earlier, default 0) (vllm/entrypoints/openai/chat_completion/protocol.py:393-402). The field description says that a value other than 0 raises an error without priority scheduling. At this commit, no code raises that error: the default first-come-first-served queue does not use the value (vllm/v1/core/sched/request_queue.py:200-205).
• Structured output: structured_outputs (vllm/entrypoints/openai/chat_completion/protocol.py:389-392).
• Bookkeeping: request_id (default: a random id), cache_salt, vllm_xargs for custom extensions (vllm/entrypoints/openai/chat_completion/protocol.py:403-410, vllm/entrypoints/openai/chat_completion/protocol.py:525-531).

Sampling defaults. to_sampling_params() uses this order for each of seven fields: the request value, else the server default, else the OpenAI default (vllm/entrypoints/openai/chat_completion/protocol.py:735-747). The OpenAI defaults are temperature 1.0, top_p 1.0, top_k 0, min_p 0.0 and penalties that do nothing (vllm/entrypoints/openai/chat_completion/protocol.py:692-700). The server default comes from ModelConfig.get_diff_sampling_param() (vllm/entrypoints/openai/chat_completion/serving.py:181):

1. With generation_config = "vllm", it reads no file. Otherwise it reads the model's generation config (vllm/config/model.py:1875-1877). The default is "auto" (vllm/config/model.py:358).
2. --override-generation-config replaces single values (vllm/config/model.py:1879-1880).
3. It keeps only nine known keys, and it renames max_new_tokens to max_tokens (vllm/config/model.py:1882-1903).
4. It logs a warning when a generation config file changes the defaults (vllm/config/model.py:1906-1915).

max_tokens. get_max_tokens() returns the smallest of these values that is set (vllm/entrypoints/serve/utils/api_utils.py:234-270):

• max_model_len − prompt length;
• the request value, or max_tokens from the server default when the request has none;
• an override: max_new_tokens from --override-generation-config when --generation-config is auto or vllm, else max_new_tokens from the generation config folder that you give (vllm/entrypoints/openai/chat_completion/serving.py:183-187);
• a platform limit.

Worked example. max_model_len is 4,096 and the rendered prompt has 13 tokens. A request without max_completion_tokens can make up to 4,096 − 13 = 4,083 tokens. With "max_completion_tokens": 32, it makes at most 32. A prompt of 4,100 tokens raises ValueError("Input length (4100) exceeds model's maximum context length (4096).") (vllm/entrypoints/serve/utils/api_utils.py:248-252). The renderer normally rejects such a prompt before this, with HTTP 400.

Start a server with vllm serve Qwen/Qwen3-0.6B. Then send this request (example commands, not in the repo). The option -N turns off the output buffer of curl, so you see each event when it comes.

curl -N http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-0.6B",
       "messages": [{"role": "user", "content": "Hi"}],
       "max_completion_tokens": 32,
       "stream": true,
       "stream_options": {"include_usage": true},
       "chat_template_kwargs": {"enable_thinking": false}}'

The reply has this form. The ids, times, text and token counts are illustrative; "..." stands for the object, created and model fields.

data: {"id":"chatcmpl-3f9c...","object":"chat.completion.chunk","created":1761140000,"model":"Qwen/Qwen3-0.6B","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"prompt_token_ids":null,"prompt_text":null}

data: {"id":"chatcmpl-3f9c...",...,"choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-3f9c...",...,"choices":[{"index":0,"delta":{"content":"!"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

...

data: {"id":"chatcmpl-3f9c...",...,"choices":[{"index":0,"delta":{"content":""},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"weight_versions":null}]}

data: {"id":"chatcmpl-3f9c...",...,"choices":[],"usage":{"prompt_tokens":13,"total_tokens":23,"completion_tokens":10},"system_fingerprint":"vllm-<version>-<hash8>"}

data: [DONE]

1. The first chunk carries only the role and empty content (vllm/entrypoints/openai/chat_completion/serving.py:590-598).
2. Each later chunk carries the new text of one engine output, in delta.content. Each chunk is serialized with exclude_unset=True, so only the fields that the code sets appear (vllm/entrypoints/openai/chat_completion/serving.py:497-503).
3. The last content chunk has finish_reason. Its content is empty because the detokenizer does not decode the end token (vllm/v1/engine/detokenizer.py:107-111).
4. The usage chunk has choices: [] and usage (vllm/entrypoints/openai/chat_completion/serving.py:854-903). With empty choices, the serializer also drops all None fields (exclude_none=not chunk.choices). The server sends it only with include_usage or --enable-force-include-usage (vllm/entrypoints/serve/utils/api_utils.py:276-285). When usage is on, the fingerprint moves from the last content chunk to this chunk (vllm/entrypoints/openai/chat_completion/serving.py:815-824).
5. data: [DONE] always ends the stream, also after an error chunk (vllm/entrypoints/openai/chat_completion/serving.py:504-508).

        try:
            async for chunk in self.chat_completion_stream_generator_raw(
                request,
                result_generator,
                request_id,
                model_name,
                conversation,
                tokenizer,
                request_metadata,
                chat_template_kwargs=chat_template_kwargs,
                mm_token_counts=mm_token_counts,
            ):
                if isinstance(chunk, ErrorResponse):
                    data = chunk.model_dump_json()
                else:
                    data = chunk.model_dump_json(
                        exclude_unset=True, exclude_none=not chunk.choices
                    )
                yield f"data: {data}\n\n"
        except Exception as e:
            logger.exception("Error serializing chat completion stream.")
            data = self.create_streaming_error_response(e)
            yield f"data: {data}\n\n"
        yield "data: [DONE]\n\n"

Check the numbers. total_tokens = 13 + 10 = 23. The 10 output tokens are nine text tokens and the end token. The 13 prompt tokens come from the Qwen3 chat template around the word "Hi" (inferred, not in the repo).

Keep-alive comments. A proxy can close an idle stream. With --sse-keep-alive-interval N (default 0, off), the server sends the comment line : keep-alive after N idle seconds (vllm/entrypoints/common/cli_args.py:156-160, vllm/entrypoints/serve/utils/sse_keep_alive.py:18-33). SSE clients ignore comment lines.

The type of one message is a union of three types (vllm/renderers/chat_utils/types.py:316-320). They are the OpenAI message types, a custom message type and the Harmony message type of gpt-oss models. The custom type has role: Required[str], so vLLM accepts any role name that the chat template knows (vllm/renderers/chat_utils/types.py:284-288). It also has the extra keys reasoning (earlier thinking text) and tools (vllm/renderers/chat_utils/types.py:306-310).

• add_generation_prompt (default true) adds the start of an assistant turn to the prompt. continue_final_message instead continues the last assistant message. A request that sends both as true gets an error (vllm/entrypoints/openai/chat_completion/protocol.py:1045-1055).
• With add_generation_prompt, the role of the answer is --response-role, default assistant. Without it, the answer takes the role of the last message (vllm/entrypoints/common/cli_args.py:109-110, vllm/entrypoints/openai/chat_completion/serving.py:440-443).

The page Renderers and chat templates: from messages to token ids explains how the renderer finds and applies the template.

Tool calls. The client sends tools (function definitions). The model writes a tool call as text in its own format. A tool parser turns that text into the OpenAI form message.tool_calls[i] = {id, type: "function", function: {name, arguments}} (vllm/entrypoints/generate/base/protocol.py:316-328).

1. For tool_choice: "auto", start the server with --enable-auto-tool-choice and --tool-call-parser <name>. Without them, an "auto" request gets the error "auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set (vllm/renderers/online_renderer.py:260-271). The flag without a parser stops start-up (vllm/entrypoints/common/cli_args.py:524-525).
2. "required" or a named function also needs both flags. The error text names only --tool-call-parser (vllm/renderers/online_renderer.py:272-281). But without --enable-auto-tool-choice, the server loads no tool parser, even when --tool-call-parser is set (vllm/parser/parser_manager.py:34-35).
3. In a stream, a tool call comes in pieces as delta.tool_calls. Each piece has an index, and the name and the arguments can come in separate chunks (vllm/entrypoints/generate/base/protocol.py:331-342).

Reasoning content. Reasoning models write their thinking before the answer, for example between <think> tags. With --reasoning-parser <name> (vllm/engine/arg_utils.py:1136-1139), vLLM moves that text out of content into the vLLM-specific field reasoning of the message or the delta (vllm/entrypoints/openai/chat_completion/protocol.py:79-80, vllm/entrypoints/generate/base/protocol.py:356-360). Without a parser, the thinking text stays in content. The request field include_reasoning (default true) controls whether the server returns it (vllm/entrypoints/openai/chat_completion/protocol.py:271).

The page Tool and reasoning parsers: from model text to tool_calls and reasoning explains the parsers and how to add your own.

Every error body has the same shape: {"error": {"message", "type", "param", "code"}} (vllm/entrypoints/serve/engine/protocol.py:79-87). The app registers its handlers in four layers (vllm/entrypoints/serve/exception_handling/register.py:26-42):

1. FastAPI errors. A body that does not match the pydantic model gives RequestValidationError. The handler answers 400 with at most 10 error entries (vllm/entrypoints/serve/exception_handling/handlers/validation.py:149-195).
2. vLLM errors. All vLLM error classes derive from VLLMError (vllm/exceptions.py:10-28). One handler, vllm_error_handler(), gets them all (vllm/entrypoints/serve/exception_handling/register.py:32, vllm/entrypoints/serve/exception_handling/handlers/vllm_error.py:18-25). create_error_response() then selects the status from the class. VLLMValidationError becomes 400 BadRequestError with param set (vllm/entrypoints/serve/exception_handling/error_response.py:43-46). VLLMUnprocessableEntityError becomes 422, and a not-found error becomes 404 (vllm/entrypoints/serve/exception_handling/error_response.py:47-54). A GenerationError keeps its own status: 500, or 503 for a retryable error (vllm/entrypoints/serve/exception_handling/error_response.py:60-67, vllm/exceptions.py:131-144). Other server errors become 500 (vllm/entrypoints/serve/exception_handling/error_response.py:68-77).
3. Raw Python errors. ValueError, TypeError and OverflowError still become 400; NotImplementedError becomes 501 (vllm/entrypoints/serve/exception_handling/error_response.py:80-87).
4. Anything else becomes 500. The one exception is a Jinja TemplateError from the chat template, which becomes 400 (vllm/entrypoints/serve/exception_handling/error_response.py:88-96).

Which handler answers a validator error. Many request checks are pydantic validators with mode="before" that raise VLLMValidationError. Examples are "Stream options can only be defined when stream=True" (vllm/entrypoints/openai/chat_completion/protocol.py:845-856) and the tool_choice checks (vllm/entrypoints/openai/chat_completion/protocol.py:959-1043). The path of such an error is not the path of a type error:

1. VLLMValidationError is not a ValueError (vllm/exceptions.py:20, vllm/exceptions.py:28). Pydantic turns only a ValueError or an AssertionError of a validator into a ValidationError, and lets other exceptions pass unchanged (general pydantic rule). The tests of vLLM show this: building the request raises VLLMValidationError itself (tests/tool_use/test_chat_completion_request_validations.py:30-38).
2. So FastAPI does not make a RequestValidationError, and the validation handler of layer 1 does not run. The exception goes to vllm_error_handler() of layer 2, which calls exception_handler() and then create_error_response() (vllm/entrypoints/serve/exception_handling/handlers/vllm_error.py:24-25, vllm/entrypoints/serve/exception_handling/handlers/exception.py:14-24).
3. The message is str(exc), and the class adds the parameter name to it (vllm/entrypoints/serve/exception_handling/error_response.py:22-24, vllm/exceptions.py:49-56). The server test expects a BadRequestError (HTTP 400) for this request (tests/entrypoints/openai/chat_completion/test_chat.py:406-415).

The body for "stream": false with stream_options is then (the field order follows ErrorInfo):

HTTP/1.1 400 Bad Request

{"error": {"message": "Stream options can only be defined when `stream=True`. (parameter=stream_options)",
           "type": "BadRequestError", "param": "stream_options", "code": 400}}

A field with a wrong type, for example "n": "x", takes the other path. Pydantic raises a ValidationError, and FastAPI makes a RequestValidationError. The validation handler answers 400 with the message "1 validation error: ..." and a param such as body.n (vllm/entrypoints/serve/exception_handling/handlers/validation.py:167-195). That handler still looks for a VLLMValidationError inside the pydantic errors (vllm/entrypoints/serve/exception_handling/handlers/validation.py:158-165). Because the class is no longer a ValueError, this branch cannot match a validator error any more (derived from the class tree).