Measured on our Macs
Specs give a ceiling; software lands below it. These are 62 runs of 38 models on three Macs we own, from the benchmark suite of Akhna, our Mac app (2026-03-31 to 2026-05-21), each next to the ceiling the bandwidth rule gives for that exact model file. measured
What they say
- Dense models reach about 78% of the bandwidth ceiling on every Mac we measured (median of 31; range 62-85%). So on a Mac, read a derived decode ceiling as roughly 78% of that in practice.
- Mixture-of-experts models reach about 41% (Qwen3-30B-A3B family, 4 rows). Their arithmetic advantage is real but about half of it survives: per token they read little, so fixed costs weigh more.
- Small models (under 3 GB) reach about 72%: at hundreds of tokens per second, per-token overhead is a large share.
- Time to first word is where Mac generations differ most. Answer speed scales with bandwidth (about 6x from the M2 Air to the M5 Max); time to first word on the same prompt shrinks about 20x. Apple publishes no compute figure, so this is only visible by measuring.
The same models on three Macs
Context: a 6,000-character prompt (the token count is in each cell), about 70-150 tokens generated, 3 repetitions, median. Ceiling = bandwidth / (bytes of the model file streamed per token + the KV cache at that length), from the file's size and config on Hugging Face. derived
| Model (4-bit MLX) | MacBook Air, M2 (10-core GPU), 24 GB 100 GB/s spec | MacBook Air, M5 (10-core GPU), 32 GB 153 GB/s spec | MacBook Pro 16, M5 Max (40-core GPU), 128 GB 614 GB/s spec |
|---|---|---|---|
| Qwen2.5-3B-Instruct-4bit 1.7 GB file |
40.0 tok/s of 56 (72%) first word 5.438 s · 1,657 tokens | 62.1 tok/s of 85 (73%) first word 3.023 s · 1,657 tokens | 210.1 tok/s of 341 (62%) first word 0.258 s · 1,657 tokens |
| Llama-3.2-3B-Instruct-4bit 1.8 GB file |
40.6 tok/s of 50 (81%) first word 3.478 s · 1,540 tokens | 61.0 tok/s of 77 (80%) first word 2.665 s · 1,540 tokens | 220.5 tok/s of 308 (72%) first word 0.232 s · 1,540 tokens |
| Phi-4-mini-instruct-4bit 2.2 GB file |
33.5 tok/s of 42 (79%) first word 5.138 s · 1,489 tokens | — | 177.4 tok/s of 260 (68%) first word 0.314 s · 1,489 tokens |
| Qwen3-4B-Instruct-2507-4bit 2.3 GB file |
30.3 tok/s of 40 (76%) first word 9.549 s · 1,657 tokens | — | 172.4 tok/s of 244 (71%) first word 0.316 s · 1,657 tokens |
| Qwen2.5-7B-Instruct-4bit 4.3 GB file |
18.8 tok/s of 24 (77%) first word 12.624 s · 1,657 tokens | 30.0 tok/s of 38 (80%) first word 6.35 s · 1,657 tokens | 117.3 tok/s of 151 (78%) runs disagree first word 0.573 s · 1,657 tokens |
| Meta-Llama-3.1-8B-Instruct-4bit 4.5 GB file |
17.9 tok/s of 23 (79%) first word 11.873 s · 1,540 tokens | 23.8 tok/s of 34 (69%) first word 8.238 s · 1,540 tokens | 110.7 tok/s of 138 (80%) first word 0.516 s · 1,540 tokens |
| Qwen2.5-14B-Instruct-4bit 8.3 GB file |
8.0 tok/s of 12 (66%) first word 24.869 s · 1,657 tokens | 14.4 tok/s of 19 (78%) runs disagree first word 16.939 s · 1,657 tokens | 57.6 tok/s of 75 (77%) first word 1.25 s · 1,657 tokens |
| Qwen2.5-32B-Instruct-4bit 18.4 GB file |
— | 5.1 tok/s of 8 (62%) first word 41.958 s · 1,657 tokens | 27.2 tok/s of 33 (82%) first word 2.546 s · 1,657 tokens |
Every model on the M5 Max
| Model | File | Answer speed | Ceiling derived | Share | First word | Prompt speed |
|---|---|---|---|---|---|---|
| Llama-3.2-1B-Instruct-4bit | 0.7 GB | 464.3 tok/s | 821 | 57% | 0.106 s | 22,587 tok/s |
| Llama-3.2-3B-Instruct-4bit | 1.8 GB | 220.5 tok/s | 308 | 72% | 0.232 s | 8,107 tok/s |
| Qwen2.5-3B-Instruct-4bit | 1.7 GB | 210.1 tok/s | 341 | 62% | 0.258 s | 7,791 tok/s |
| Phi-4-mini-instruct-4bit | 2.2 GB | 177.4 tok/s | 260 | 68% | 0.314 s | 6,102 tok/s |
| gemma-3-text-4b-it-4bit | 2.6 GB | 173.5 tok/s | 247 | 70% | 0.379 s | 6,088 tok/s |
| Qwen3-4B-Instruct-2507-4bit | 2.3 GB | 172.4 tok/s | 244 | 71% | 0.316 s | 6,174 tok/s |
| Qwen3-30B-A3B-4bit (MoE) | 17.2 GB | 135.5 tok/s | 326 | 42% | 0.399 s | 4,685 tok/s |
| gpt-oss-20b-MXFP4-Q8 (MoE) | 12.1 GB | 134.3 tok/s | not computed | — | 0.371 s | 5,087 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit (MoE) | 17.2 GB | 132.5 tok/s | 325 | 41% | 0.395 s | 4,727 tok/s |
| Qwen3-Coder-30B-A3B-Instruct-4bit (MoE) | 17.2 GB | 132.5 tok/s | 325 | 41% | 0.391 s | 4,782 tok/s |
| Qwen2.5-7B-Instruct-4bit | 4.3 GB | 117.3 tok/s runs disagree | 151 | 78% | 0.573 s | 3,167 tok/s |
| Meta-Llama-3.1-8B-Instruct-4bit | 4.5 GB | 110.7 tok/s | 138 | 80% | 0.516 s | 3,208 tok/s |
| DeepSeek-R1-Distill-Llama-8B-4bit | 4.5 GB | 108.9 tok/s | 139 | 78% | 0.493 s | 3,298 tok/s |
| Hermes-3-Llama-3.1-8B-4bit | 4.5 GB | 108.5 tok/s | 139 | 78% | 0.459 s | 3,587 tok/s |
| aya-expanse-8b-4bit | 4.5 GB | 101.3 tok/s | 148 | 69% | 0.827 s | 2,322 tok/s |
| granite-3.3-8b-instruct-4bit | 4.6 GB | 95.8 tok/s | 125 | 77% | 0.623 s | 3,129 tok/s |
| Qwen3-30B-A3B-Instruct-2507-8bit (MoE) | 32.4 GB | 93.9 tok/s | 180 | 52% | 0.467 s | 3,926 tok/s |
| Yi-1.5-9B-Chat-4bit | 5.0 GB | 89.2 tok/s | 123 | 73% | 0.657 s | 2,921 tok/s |
| falcon3-10b-instruct-4bit | 5.8 GB | 75.6 tok/s | 104 | 72% | 1.154 s | 1,588 tok/s |
| Mistral-Nemo-Instruct-2407-4bit | 6.9 GB | 71.8 tok/s | 90 | 80% | 0.804 s | 2,234 tok/s |
| Qwen2.5-7B-Instruct-8bit | 8.1 GB | 65.2 tok/s | 81 | 81% | 0.517 s | 3,509 tok/s |
| Qwen2.5-14B-Instruct-4bit | 8.3 GB | 57.6 tok/s | 75 | 77% | 1.25 s | 1,376 tok/s |
| Phi-3.5-mini-instruct-bf16 | 7.6 GB | 57.4 tok/s | 74 | 77% | 0.449 s | 4,594 tok/s |
| Reka-Flash-3-4bit | 11.8 GB | 42.4 tok/s | 53 | 80% | 1.77 s | 933 tok/s |
| Mistral-Small-3.2-24B-Instruct-2506-4bit | 13.3 GB | 39.6 tok/s | 47 | 85% | 1.769 s | 990 tok/s |
| QwQ-32B-Preview-4bit | 18.4 GB | 27.8 tok/s | 33 | 84% | 2.712 s | 622 tok/s |
| c4ai-command-r-08-2024-4bit | 18.2 GB | 27.3 tok/s | 36 | 77% | 3.273 s | 543 tok/s |
| Qwen2.5-32B-Instruct-4bit | 18.4 GB | 27.2 tok/s | 33 | 82% | 2.546 s | 663 tok/s |
| gemma-3-27b-it-4bit | 16.8 GB | 26.8 tok/s | 35 | 77% | 3.007 s | 579 tok/s |
| Yi-1.5-34B-Chat-4bit | 19.3 GB | 26.4 tok/s | 31 | 84% | 2.75 s | 680 tok/s |
| Mixtral-8x22B-Instruct-v0.1-4bit (MoE) | 79.1 GB | 22.1 tok/s | not computed | — | 4.481 s | 433 tok/s |
| Llama-3.1-Nemotron-70B-Instruct-HF-4bit | 39.7 GB | 13.1 tok/s | 16 | 85% | 4.468 s | 343 tok/s |
| Athene-V2-Chat-4bit | 40.9 GB | 12.7 tok/s | 15 | 84% | 5.658 s | 295 tok/s |
| Qwen2.5-72B-Instruct-4bit | 40.9 GB | 12.6 tok/s | 15 | 83% | 5.869 s | 285 tok/s |
| Llama-3.3-70B-Instruct-4bit | 39.7 GB | 12.1 tok/s | 16 | 78% | 6.207 s | 250 tok/s |
| Meta-Llama-3.1-70B-Instruct-4bit | 39.7 GB | 12.0 tok/s | 16 | 78% | 6.452 s | 243 tok/s |
| c4ai-command-r-plus-08-2024-4bit | 58.4 GB | 8.9 tok/s | 11 | 83% | 9.715 s | 178 tok/s |
| Mistral-Large-Instruct-2407-4bit | 69.0 GB | 7.4 tok/s | 9 | 84% | 12.634 s | 153 tok/s |
"Not computed": a mixture-of-experts file whose config does not state its expert shape (Mixtral, gpt-oss), so the bytes read per token are not known from the file.
Long prompts on the M5 Max
Reading a long prompt grows faster than its length (attention), and answer speed falls as the cache grows. This is the wait in document Q&A.
| Model | Prompt tokens | First word | Prompt speed | Answer speed |
|---|---|---|---|---|
| Llama-3.3-70B-Instruct-4bit | 13,377 | 50.9 s | 264 tok/s | 12.8 tok/s |
| Llama-3.3-70B-Instruct-4bit | 17,817 | 72.2 s | 247 tok/s | 12.3 tok/s |
| Llama-3.3-70B-Instruct-4bit | 26,707 | 116.0 s | 230 tok/s | 11.5 tok/s |
| Llama-3.3-70B-Instruct-4bit | 35,597 | 163.1 s | 218 tok/s | 10.5 tok/s |
| Qwen2.5-14B-Instruct-4bit | 13,356 | 11.6 s | 1,159 tok/s | 47.3 tok/s |
| Qwen2.5-14B-Instruct-4bit | 17,796 | 19.0 s | 943 tok/s | 36.0 tok/s |
| Qwen2.5-14B-Instruct-4bit | 26,686 | 32.2 s | 832 tok/s | 39.6 tok/s |
| Qwen2.5-14B-Instruct-4bit | 35,576 | 43.6 s | 818 tok/s | 26.1 tok/s |
| Qwen2.5-14B-Instruct-4bit | 53,356 | 77.5 s | 690 tok/s | 29.2 tok/s |
| Qwen2.5-14B-Instruct-4bit | 71,136 | 122.4 s | 582 tok/s | 23.9 tok/s |
| Qwen2.5-32B-Instruct-4bit | 13,356 | 26.6 s | 503 tok/s | 25.1 tok/s |
| Qwen2.5-32B-Instruct-4bit | 17,796 | 36.4 s | 490 tok/s | 23.8 tok/s |
| Qwen2.5-32B-Instruct-4bit | 26,686 | 57.9 s | 461 tok/s | 22.1 tok/s |
| Qwen2.5-32B-Instruct-4bit | 35,576 | 85.1 s | 419 tok/s | 15.9 tok/s |
| Qwen2.5-32B-Instruct-4bit | 53,356 | 151.7 s | 352 tok/s | 16.6 tok/s |
| Qwen2.5-32B-Instruct-4bit | 71,136 | 218.9 s | 325 tok/s | 14.7 tok/s |
| Qwen2.5-7B-Instruct-4bit | 13,356 | 4.0 s | 3,371 tok/s | 106.9 tok/s |
| Qwen2.5-7B-Instruct-4bit | 17,796 | 6.1 s | 2,941 tok/s | 99.0 tok/s |
| Qwen2.5-7B-Instruct-4bit | 26,686 | 10.9 s | 2,470 tok/s | 92.6 tok/s |
| Qwen2.5-7B-Instruct-4bit | 35,576 | 16.6 s | 2,158 tok/s | 69.3 tok/s |
| Qwen2.5-7B-Instruct-4bit | 53,356 | 31.1 s | 1,720 tok/s | 68.4 tok/s |
| Qwen2.5-7B-Instruct-4bit | 71,136 | 51.0 s | 1,398 tok/s | 56.1 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 13,356 | 3.6 s | 3,744 tok/s | 98.4 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 17,796 | 5.8 s | 3,096 tok/s | 89.0 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 26,686 | 11.0 s | 2,437 tok/s | 76.3 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 35,576 | 17.1 s | 2,088 tok/s | 57.5 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 53,356 | 33.9 s | 1,578 tok/s | 53.0 tok/s |
| Qwen3-30B-A3B-Instruct-2507-4bit | 71,136 | 57.8 s | 1,233 tok/s | 43.2 tok/s |
How these were run
- Machines: MacBook Air, M2 (10-core GPU), 24 GB (macOS 26.2); MacBook Air, M5 (10-core GPU), 32 GB (macOS 26.3.2); MacBook Pro 16, M5 Max (40-core GPU), 128 GB (macOS 26.4). Runtime: MLX (mlx-lm); models are the mlx-community files named.
- Each run: prompts of 500 to 12,000 characters, 3 repetitions each; we show the median. Where several runs exist, we show the fastest run and flag "runs disagree" when they differ by more than 15%: other work on the machine can only slow a run down.
- Dated: macOS and MLX have moved since these runs (2026-05-21). A re-run on the current versions is planned; until then, read them as of their dates.
- Source: the benchmark suite of Akhna, our Mac app; model sizes and shapes from Hugging Face.