Mistral AI engineers uncover real cause of memory leak in vLLM using BPFtrace
Mistral
Mistral AI engineers investigated a memory leak in vLLM during disaggregated serving with Mistral Medium 3.1. After using Python tools, Heaptrack, and BPFtrace, they found the leak was caused by direct mmap calls via glibc's syscall wrapper, not heap allocations.
Mistral AI's team investigated a suspected memory leak in vLLM during pre-production testing of disaggregated serving with their frontier model Mistral Medium 3.1. The leak manifested as a steady 400 MB per minute increase in system memory under specific conditions: with vLLM, Mistral Medium 3.1, and graph compilation enabled, only on the decode side of the Prefill/Decode disaggregated setup using NIXL. Initial Python tools like Memray and Guppy 3 showed no leak, and Heaptrack only revealed peak RSS discrepancy. By using pmap to monitor /proc/<pid>/maps, they observed growing anonymous memory mappings with changing addresses, characteristic of mremap or repeated mmap/munmap cycles. To pinpoint the exact cause, they used BPFtrace to trace system calls, revealing that the allocations were done via direct mmap calls through glibc's syscall wrapper (syscall+29). This showed the leak was outside heap management, requiring low-level tracing to identify.
- Сокращения
- vLLM = Virtual Large Language Model — виртуальная большая языковая модель
- RSS = Resident Set Size — размер резидентного набора
- PID = Process ID — идентификатор процесса
- BPFtrace = BPF trace — трассировка BPF
- NIXL = NIXL — NIXL
- UCX = Unified Communication X — Unified Communication X
- HPC = High-Performance Computing — высокопроизводительные вычисления
- GDB = GNU Debugger — отладчик GNU
- malloc = memory allocation — выделение памяти
- free = free memory — освобождение памяти
- mmap = memory map — отображение памяти
- mremap = memory remap — переотображение памяти
- munmap = memory unmap — отмена отображения памяти
- sbrk = set break — установка разрыва
- brk = break — разрыв
- eBPF = extended Berkeley Packet Filter — расширенный фильтр пакетов Беркли
- PTRACE = process trace — трассировка процесса
- LD_PRELOAD = load preload — загрузка предварительной загрузки
Source: Mistral AI —
original