Macs

Which AI models run on the M2 Ultra (128 GB, 60-core GPU)?

The M2 Ultra (128 GB, 60-core GPU) has 128 GB of memory, about 96 GB of it usable for a model, and 800 GB/s of memory bandwidth. 101 of the 110 models in our catalog run on it at 4-bit or better.

Data as of 2 November 2026 · Vendor specs

Compare with other machines

128 GBmemory ~96 GB usable for a model

800 GB/sbandwidth, sets the speed

101models run at 4-bit or better

Our pick for this machine Nemotron 3.5 Super VL 120B-A12B Q4_K_M · 76 of 96 GB · ~60 tok/s our est. The largest model on our shortlist of current models that runs at 4-bit or better and 10 tok/s or more.

Models that run on it

Model	Best build	Memory	Speed
Nemotron 3 Super 120B-A12B120B MoE	Q4_K_M	90 GB	~37 tok/s ModelFit est.
Nemotron 3.5 Super VL 120B-A12B120B MoE	Q4_K_M	76 GB	~60 tok/s our est.
gpt-oss-120b117B MoE	MXFP4	68 GB	~90 tok/s ModelFit est.
Llama 3.3 70B70.6B	Q6_K	60 GB	~10 tok/s ModelFit est.
Llama 3.3 Nemotron Super 49B49B	Q6_K	42 GB	~11 tok/s our est.
Qwen3.6-35B-A3B35B MoE	Q8_0	41 GB	~99 tok/s ModelFit est.
Gemma 4 31B30.7B	Q8_0	33 GB	~17 tok/s ModelFit est.
Qwen3-Coder-30B-A3B30.5B MoE	Q8_0	34 GB	~80 tok/s ModelFit est.
Nemotron 3 Nano 30B-A3B30B MoE	FP16	62 GB	~73 tok/s our est.
Qwen3.8-27B27.3B	Q8_0	29 GB	~21 tok/s ModelFit est.
Gemma 4 26B-A4B25.2B MoE	Q8_0	30 GB	~76 tok/s ModelFit est.
gpt-oss-20b21B MoE	MXFP4	15.1 GB	~103 tok/s ModelFit est.
Nemotron Nano 9B v29B	FP16	19.4 GB	~24 tok/s our est.
Qwen3.5 9B Instruct9B	Q8_0	12.0 GB	~59 tok/s ModelFit est.
Qwen2.5 7B Instruct7B	Q4_K_M	6.6 GB	~95 tok/s ModelFit est.
Gemma 3 4B Instruct4B	Q8_0	5.0 GB	~98 tok/s ModelFit est.
Ornith 1.5 122B125.09B	Q4_K_M	79 GB	~66 tok/s ModelFit est.
Ling-3.0-flash124B	Q4_K_M	81 GB	~94 tok/s ModelFit est.
Devstral 2 123B123B	Q4_K_M	78 GB	~8.0 tok/s slow ModelFit est.
Qwen3.5 122B-A10B Instruct122B MoE	Q4_K_M	75 GB	~62 tok/s ModelFit est.
Llama 4 Scout109B MoE	Q4_K_M	70 GB	~45 tok/s ModelFit est.
Qwen3-Next 80B-A3B80B MoE	Q8_0	88 GB	~124 tok/s ModelFit est.
Kolibri 178.1B	Q4_K_M	52 GB	~112 tok/s ModelFit est.
Llama 3.1 70B Instruct70B	Q4_K_M	44 GB	~14 tok/s ModelFit est.
DeepSeek-R1 Distill Llama 70B70B	Q4_K_M	44 GB	~14 tok/s ModelFit est.
Mixtral 8x7B Instruct46.7B MoE	Q4_K_M	32 GB	~43 tok/s ModelFit est.
Flint 40B-A2.5B39.8B MoE	NVFP4	24 GB	~146 tok/s ModelFit est.
Sage-A1.535.95B	Q4_K_M	23 GB	~139 tok/s ModelFit est.
Qwen3.5 35B-A3B Instruct35B MoE	Q8_0	41 GB	~99 tok/s ModelFit est.
Ornith 1.0 35B35B	Q4_K_M	23 GB	~141 tok/s ModelFit est.
Nanbeige4.3 33B33.1B	Q8_0	37 GB	~17 tok/s ModelFit est.
Poolside Laguna XS.233B	Q4_K_M	25 GB	no estimate
Laguna XS 2.133B	Q4_K_M	22 GB	~133 tok/s ModelFit est.
Qwen3 32B32B	Q8_0	37 GB	~17 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 32B32B	Q8_0	37 GB	~17 tok/s ModelFit est.
Olmo 3.1 32B Instruct32B	Q4_K_M	21 GB	~28 tok/s ModelFit est.
Qwen3 30B30B	Q8_0	32 GB	~80 tok/s ModelFit est.
NVIDIA Nemotron Cascade 2 30B-A3B30B MoE	Q6_K	26 GB	~183 tok/s our est.
Cohere North Mini Code30B	Q4_K_M	20 GB	no estimate
Nemotron 3.5 Lightning 30B-A3B30B MoE	Q4_K_M	27 GB	~99 tok/s ModelFit est.
GLM-4.7-Flash 30B-A3B30B MoE	Q4_K_M	20 GB	~108 tok/s ModelFit est.
Granite Switch 4.1 30B30B	Q4_K_M	23 GB	~26 tok/s ModelFit est.
Muse Glimmer 30B29.8B	Q4_K_M	18.3 GB	~34 tok/s ModelFit est.
Granite 4.2 30B29.3B	Q4_K_M	17.9 GB	~30 tok/s ModelFit est.
AREX-2 27B27.4B	Q4_K_M	18.6 GB	~33 tok/s ModelFit est.
Gemma 2 27B Instruct27B	Q4_K_M	23 GB	~31 tok/s ModelFit est.
Gemma 3 27B Instruct27B	Q4_K_M	23 GB	~32 tok/s ModelFit est.
Qwen3.5 27B Instruct27B	Q8_0	29 GB	~21 tok/s ModelFit est.
Qwen3.6 27B27B	Q8_0	32 GB	~21 tok/s ModelFit est.
Jebadiah 27B27B	Q8_0	31 GB	~21 tok/s ModelFit est.
Mistral Small 3.124B	Q8_0	25 GB	~22 tok/s ModelFit est.
LFM2 24B-A2B Instruct24B MoE	Q4_K_M	15.3 GB	~148 tok/s ModelFit est.
Devstral Small 2 24B24B	Q4_K_M	16.6 GB	~36 tok/s ModelFit est.
Mistral Small 22B22B	Q4_K_M	18.4 GB	~40 tok/s ModelFit est.
Qwen2.5 14B Instruct14B	Q4_K_M	12.3 GB	~55 tok/s ModelFit est.
Phi-3 Medium 14B14B	Q4_K_M	12.3 GB	~40 tok/s our est.
Phi-4 14B14B	Q8_0	15.9 GB	~35 tok/s ModelFit est.
Qwen2.5 Coder 14B14B	Q4_K_M	12.3 GB	~55 tok/s ModelFit est.
Qwen3 14B14B	Q8_0	17.3 GB	~35 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 14B14B	Q8_0	16.0 GB	~35 tok/s ModelFit est.
Ministral 3 14B13.5B	Q4_K_M	10.3 GB	~55 tok/s ModelFit est.
Mellum2.1 12B-A2.5B Thinking12.1B MoE	Q4_K_M	9.3 GB	~263 tok/s our est.
Mistral Nemo 12B12B	Q8_0	13.4 GB	~41 tok/s ModelFit est.
Gemma 3 12B Instruct12B	Q4_K_M	10.7 GB	~63 tok/s ModelFit est.
Gemma 4 12B12B	Q8_0	14.1 GB	~41 tok/s ModelFit est.
GLM-4-9B-Chat9.4B	Q8_0	11.2 GB	~54 tok/s ModelFit est.
Gemma 2 9B Instruct9B	Q4_K_M	8.2 GB	~67 tok/s ModelFit est.
Ornith 1.0 9B9B	Q4_K_M	6.7 GB	~88 tok/s ModelFit est.
MiMo-V2.6-Distill-Qwen-9B9B	Q4_K_M	6.9 GB	~86 tok/s ModelFit est.
Jebadiah 9B9B	Q8_0	11.0 GB	~58 tok/s ModelFit est.
Granite 4.2 8B8.8B	Q4_K_M	6.1 GB	~80 tok/s ModelFit est.
Granite Switch 4.1 8B8.8B	Q4_K_M	7.6 GB	~68 tok/s ModelFit est.
MiniCPM-V 4.5 8B8.7B	Q4_K_M	6.8 GB	~86 tok/s ModelFit est.
Ministral 3 8B8.4B	Q4_K_M	7.1 GB	~76 tok/s ModelFit est.
LFM2.5 8B-A1B8.3B MoE	Q4_K_M	6.6 GB	~172 tok/s ModelFit est.
Llama 3.1 8B Instruct8B	Q8_0	9.2 GB	~59 tok/s ModelFit est.
Qwen3 8B8B	Q8_0	9.3 GB	~58 tok/s ModelFit est.
Granite 4.1 8B Instruct8B	Q4_K_M	6.6 GB	~77 tok/s ModelFit est.
Mistral 7B Instruct7B	Q8_0	8.4 GB	~61 tok/s ModelFit est.
Qwen2.5 Coder 7B7B	Q4_K_M	6.6 GB	~95 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 7B7B	Q8_0	8.7 GB	~64 tok/s ModelFit est.
Gemma 4 E4B4.5B	Q8_0	8.7 GB	~88 tok/s ModelFit est.
Qwen3.5 4B Instruct4B	Q8_0	5.4 GB	~94 tok/s ModelFit est.
NVIDIA Nemotron 3.5 Bolt E4B4B	Q4_K_M	9.1 GB	~106 tok/s ModelFit est.
Phi-3 Mini 3.8B3.8B	Q4_K_M	4.3 GB	~138 tok/s our est.
Phi-4 Mini 3.8B3.8B	Q8_0	4.9 GB	~94 tok/s ModelFit est.
Granite 4.2 3B3.7B	Q4_K_M	3.2 GB	~141 tok/s ModelFit est.
Ministral 3 3B3.4B	Q4_K_M	4.1 GB	~116 tok/s ModelFit est.
Qwen2.5 3B Instruct3B	Q4_K_M	3.6 GB	~154 tok/s ModelFit est.
Llama 3.2 3B Instruct3B	Q4_K_M	3.6 GB	~139 tok/s ModelFit est.
Granite 4.1 3B Instruct3B	Q4_K_M	3.0 GB	~141 tok/s ModelFit est.
Gemma 4 E2B2.3B	Q8_0	5.7 GB	~154 tok/s ModelFit est.
Gemma 2 2B Instruct2B	Q4_K_M	2.8 GB	~151 tok/s ModelFit est.
Qwen3.5 2B Instruct2B	Q4_K_M	2.8 GB	~155 tok/s ModelFit est.
LFM3 1.7B1.73B	Q4_K_M	2.1 GB	~200 tok/s ModelFit est.
Qwen2.5 1.5B Instruct1.5B	Q4_K_M	2.5 GB	~203 tok/s ModelFit est.
Llama 3.2 1B Instruct1B	Q4_K_M	2.0 GB	~215 tok/s ModelFit est.
Gemma 3 1B Instruct1B	Q8_0	2.0 GB	~204 tok/s ModelFit est.
Qwen3.5 0.8B Instruct0.8B	Q4_K_M	1.8 GB	~221 tok/s ModelFit est.
Qwen2.5 0.5B Instruct0.5B	Q4_K_M	1.8 GB	~260 tok/s ModelFit est.
SmolLM2 360M0.36B	Q4_K_M	1.5 GB	~218 tok/s ModelFit est.

Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method

Fits only at 3-bit or lower

Qwen3.8-Flash-Next.

Too big for this machine

Laguna S 2.1 (99 GB), Qwen3-235B-A22B (134 GB), DeepSeek-V4-Flash (177 GB), GLM-5.3-Flash (200 GB), Llama 3.1 405B Instruct (250 GB), Llama 4 Maverick (252 GB), Ling-3.1-flash-1019 (368 GB), DeepSeek-R1 671B (390 GB).

Questions

What is the best AI model for the M2 Ultra (128 GB, 60-core GPU)?

Nemotron 3.5 Super VL 120B-A12B at Q4_K_M: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 60 tok/s, our est.).

How much memory can a model use on the M2 Ultra (128 GB, 60-core GPU)?

About 96 GB of the 128 GB.

How fast is the M2 Ultra (128 GB, 60-core GPU) for local LLMs?

Decode speed is limited by memory bandwidth: 800 GB/s. A model that reads 10 GB per token runs at under 80 tokens per second.