Executive brief
vLLM is a high-throughput engine for serving large language models. A vulnerability in its speculative decoding feature allows a remote attacker to crash the engine by sending a specific sequence of requests. This results in a service-wide denial of service, interrupting all active users and preventing new requests until the system is manually restarted.
Technical details
A vulnerability exists in vLLM's speculative decoding implementation, specifically within the rejection sampling and next-token preparation logic. When processing certain multi-request workloads, the rejection sampler can produce a recovered token ID equal to the model's 'vocab_size'. This out-of-vocabulary value is incorrectly converted to -1 and reinjected into the live model input IDs for the next drafting step. When the drafting or attention path (e.g., flash_attn_varlen_func) later consumes this invalid token, it triggers a CUDA device-side assertion, crashing the entire vLLM worker. The issue is reachable via public gRPC interfaces using a sequence of Generate and Abort requests. This was patched in version 0.24.0.
Affected products
- vLLM Project vLLM >= 0.17.1, < 0.24.0
Timeline
- 2026-07-02: disclosed
- 2026-07-17: advisory: GitHub Advisory published
- 2026-07-17: patched