vLLM denial of service via prompt embeds on M-RoPE models
Summary
_Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server._
Sending a pure prompt embeds payload in a /v1/completions request with a model using M-RoPE causes the EngineCore to fail an assertion and fatally crash, shutting down the entire server application.
Any remote user who is authorized to make a /v1/completions endpoint can trivially make such a request and induce a crash.
Details
_Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer._
In commit [56669c1](https://github.com/vllm-project/vllm/commit/56669c1f293d5c53b6a19ddf2f78802fa9fff2c2), a simple assert intended to be a type-narrowing assert was added to the _init_mrope_positions method in GPUModelRunner (the offending line on main at the time of writing: https://github.com/vllm-project/vllm/blob/2d481f8a946ee0521872af0f098674a8ee01ce4a/vllm/v1/worker/gpu_model_runner.py#L1588-L1607).
```python
assert req_state.prompt_token_ids is not None, (
"M-RoPE requires prompt_token_ids to be available."
)
```
This type narrowing assert is to prevent mypy errors later in the function because None is not a valid type for mrope_model.get_mrope_input_positions. Unfortunately, this assertion is not always true. /v1/completions requests that specify prompt=None and prompt_embeds=<not none> will indeed create a CachedRequestState where prompt_token_ids is None. This triggers the assertion, which in turn crashes the EngineCore and the Server application.
```
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3997, in execute_model
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] deferred_state_corrections_fn = self._update_states(scheduler_output)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1239, in _update_states
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] self._init_mrope_positions(req_state)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1582, in _init_mrope_positions
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] assert req_state.prompt_token_ids is not None, (
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] AssertionError: M-RoPE requires prompt_token_ids to be available.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] AsyncLLM output_handler failed.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] Traceback (most recent call last):
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] outputs = await engine_core.get_output_async()
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1030, in get_output_async
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] raise self._format_exception(outputs) from None
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
```
All requests using the /v1/chat/completions endpoints will have text/prompt_token_ids parts (corresponding to the chat template), and prompt_embeds parts are handled as mm_features. This method (rightly) filters out those prompt_embeds content parts as they are treated as text positions.
A sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:
```python
def _init_mrope_positions(self, req_state: CachedRequestState):
model = self.get_model()
assert supports_mrope(model), "M-RoPE support is not implemented."
mrope_model = cast(SupportsMRoPE, model)
# Filter out prompt_embeds modality (text-only position info)
mrope_features = [
f for f in req_state.mm_features if f.modality != "prompt_embeds"
]
# Handle both token_ids and embeddings-only inputs
if req_state.prompt_token_ids is not None:
input_tokens = req_state.prompt_token_ids
e
