Executive brief
The Linux kernel's RDMA/mlx5 driver contains a flaw in how it calculates Receive Queue (RQ) Work Queue Element (WQE) sizes based on user-supplied input. An attacker can provide crafted shift values that cause undefined integer overflow behavior, potentially leading to kernel instability or memory corruption affecting systems using Mellanox RDMA adapters.
Technical details
The vulnerability exists in the set_rq_size() function in drivers/infiniband/hw/mlx5/qp.c, which computes RQ WQE size using the expression "1 << rq_wqe_shift" where rq_wqe_shift comes from untrusted user input via the ucmd structure. The original code only checked if the shift value exceeded 32 bits, allowing shifts of exactly 31–32 to pass through, which overflow a signed 32-bit integer and trigger undefined behavior. The fix replaces the unsafe left-shift operation with check_shl_overflow(), which safely validates the operation before execution. This vulnerability affects all systems using the mlx5 RDMA driver and requires a kernel patch to resolve.
Affected products
- Linux Linux kernel multiple versions (see stable tree branches)
Timeline
- 2026-08-15: disclosed: Published on NVD
- 2026-06-11: patched: Fix committed by Maher Sanalla
- 2026-07-24: patched: Fix backported to stable kernel branches