Executive brief
The Nouveau GPU driver for Linux contains a memory management bug in its device-private memory migration feature. When migrating GPU memory back to system RAM using large pages, the driver fails to properly release the memory mappings, leaking portions of the IOMMU translation tables. This could degrade system performance or cause memory leaks over time.
Technical details
The vulnerability is a resource leak in drm/nouveau/dmem.c affecting device-private memory migration with transparent huge pages (THP). When mapping large folios for migration, the code correctly records the mapped size (page_size() which is PAGE_SIZE << order for compound folios), but two error-path unmaps in nouveau_dmem_migrate_to_ram() and nouveau_dmem_migrate_copy_one() use a hardcoded PAGE_SIZE instead of the recorded dma_info.size. For large folios (order > 0), this causes a partial unmap, leaving IOMMU/IOVA mappings allocated. The fix changes both dma_unmap_page() calls to use dma_info.size, matching the approach already used in deux_dmem_migrate_chunk() and nouveau_dmem_evict_chunk(). No authentication or network access is required; the leak occurs during normal GPU memory migration operations on systems with THP enabled.
Affected products
- Linux Linux Kernel multiple versions from linux-2.6.11 through linux-7.2 (Nouveau GPU driver component)
Timeline
- 2026-09-16: disclosed: CVE-2026-89804 published on NVD
- 2026-09-01: patched: Upstream fix commit caa1bc2a0a6ca19 merged by Danilo Krummrich
- 2026-08-11: other: Fix authored by Zhenhao Wan