Mac mini M4 Real-World Tokens Per Second: Benchmark Results
Apple
Meta
Mistral
Alibaba/Qwen
DeepSeek
A Mac rental service owner benchmarked six LLMs on a Mac mini M4 (16GB) using a consistent methodology, finding generation speed is memory-bound and scales inversely with model size. Llama 3.2 3B hit 46.7 tok/s, while Qwen 2.5 14B managed only 11.7 tok/s; prompt processing was 10-20x faster. Practical advice: 8B models are comfortable on 16GB, larger models need more memory.
A Mac rental service owner conducted reproducible benchmarks on a Mac mini M4 with 16GB unified memory and 120 GB/s bandwidth, using Ollama 0.31.2, macOS 15.3.1, Q4_K_M quantization, a fixed prompt, and three runs per model (plus a discarded warm-up). Results: Llama 3.2 3B generated 46.7 tok/s, Mistral 7B 22.8, Qwen 2.5 7B 22.3, Llama 3.1 8B 21.2, DeepSeek R1 8B 20.0, and Qwen 2.5 14B 11.7. Prompt processing speeds ranged from 531 to 1720 tok/s. The author concludes generation is memory-bound, so speed roughly equals bandwidth divided by model size (e.g., ~25 tok/s predicted for 8B Q4, 21 measured). They note M4 Pro and M4 Max should scale with bandwidth, not cores. They recommend 8B models for 16GB as comfortable (20+ tok/s), while 14B feels slow; longer contexts are cheap to process but expensive to generate. Limits: single config, single quantization, text generation only; no batching tested. Data is open under CC BY.
- Abbreviations
- GB = Gigabyte — гигабайт
- CC BY = Creative Commons Attribution — лицензия Creative Commons с указанием авторства
Source: Habr — хаб ИИ —
original