Macs

Which AI models run on the M2 Ultra (96 GB)?

The M2 Ultra (96 GB) has 96 GB of memory, about 72 GB of it usable for a model, and 800 GB/s of memory bandwidth. 97 of the 110 models in our catalog run on it at 4-bit or better.

Data as of 2 November 2026 · Vendor specs

Compare with other machines

96 GBmemory ~72 GB usable for a model

800 GB/sbandwidth, sets the speed

97models run at 4-bit or better

Our pick for this machine Nemotron 3 Super 120B-A12B IQ4_XS · 67 of 72 GB · ~68 tok/s our est. The largest model on our shortlist of current models that runs at 4-bit or better and 10 tok/s or more.

Models that run on it

Model	Best build	Memory	Speed
Nemotron 3 Super 120B-A12B120B MoE	IQ4_XS	67 GB	~68 tok/s our est.
gpt-oss-120b117B MoE	MXFP4	68 GB	~90 tok/s ModelFit est.
Llama 3.3 70B70.6B	Q6_K	60 GB	~10 tok/s ModelFit est.
Llama 3.3 Nemotron Super 49B49B	Q6_K	42 GB	~11 tok/s our est.
Qwen3.6-35B-A3B35B MoE	Q8_0	41 GB	~99 tok/s ModelFit est.
Gemma 4 31B30.7B	Q8_0	33 GB	~17 tok/s ModelFit est.
Qwen3-Coder-30B-A3B30.5B MoE	Q8_0	34 GB	~80 tok/s ModelFit est.
Nemotron 3 Nano 30B-A3B30B MoE	FP16	62 GB	~73 tok/s our est.
Qwen3.8-27B27.3B	Q8_0	29 GB	~21 tok/s ModelFit est.
Gemma 4 26B-A4B25.2B MoE	Q8_0	30 GB	~76 tok/s ModelFit est.
gpt-oss-20b21B MoE	MXFP4	15.1 GB	~103 tok/s ModelFit est.
Nemotron Nano 9B v29B	FP16	19.4 GB	~24 tok/s our est.
Qwen3.5 9B Instruct9B	Q8_0	12.0 GB	~59 tok/s ModelFit est.
Qwen2.5 7B Instruct7B	Q4_K_M	6.6 GB	~95 tok/s ModelFit est.
Gemma 3 4B Instruct4B	Q8_0	5.0 GB	~98 tok/s ModelFit est.
Ling-3.0-flash124B	IQ4_XS	71 GB	no estimate
Llama 4 Scout109B MoE	Q4_K_M	70 GB	~45 tok/s ModelFit est.
Qwen3-Next 80B-A3B80B MoE	Q4_K_M	53 GB	~124 tok/s ModelFit est.
Kolibri 178.1B	Q4_K_M	52 GB	~112 tok/s ModelFit est.
Llama 3.1 70B Instruct70B	Q4_K_M	44 GB	~14 tok/s ModelFit est.
DeepSeek-R1 Distill Llama 70B70B	Q4_K_M	44 GB	~14 tok/s ModelFit est.
Mixtral 8x7B Instruct46.7B MoE	Q4_K_M	32 GB	~43 tok/s ModelFit est.
Flint 40B-A2.5B39.8B MoE	NVFP4	24 GB	~146 tok/s ModelFit est.
Sage-A1.535.95B	Q4_K_M	23 GB	~139 tok/s ModelFit est.
Qwen3.5 35B-A3B Instruct35B MoE	Q8_0	41 GB	~99 tok/s ModelFit est.
Ornith 1.0 35B35B	Q4_K_M	23 GB	~141 tok/s ModelFit est.
Nanbeige4.3 33B33.1B	Q8_0	37 GB	~17 tok/s ModelFit est.
Poolside Laguna XS.233B	Q4_K_M	25 GB	no estimate
Laguna XS 2.133B	Q4_K_M	22 GB	~133 tok/s ModelFit est.
Qwen3 32B32B	Q8_0	37 GB	~17 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 32B32B	Q8_0	37 GB	~17 tok/s ModelFit est.
Olmo 3.1 32B Instruct32B	Q4_K_M	21 GB	~28 tok/s ModelFit est.
Qwen3 30B30B	Q8_0	32 GB	~80 tok/s ModelFit est.
NVIDIA Nemotron Cascade 2 30B-A3B30B MoE	Q6_K	26 GB	~183 tok/s our est.
Cohere North Mini Code30B	Q4_K_M	20 GB	no estimate
Nemotron 3.5 Lightning 30B-A3B30B MoE	Q4_K_M	27 GB	~99 tok/s ModelFit est.
GLM-4.7-Flash 30B-A3B30B MoE	Q4_K_M	20 GB	~108 tok/s ModelFit est.
Granite Switch 4.1 30B30B	Q4_K_M	23 GB	~26 tok/s ModelFit est.
Muse Glimmer 30B29.8B	Q4_K_M	18.3 GB	~34 tok/s ModelFit est.
Granite 4.2 30B29.3B	Q4_K_M	17.9 GB	~30 tok/s ModelFit est.
AREX-2 27B27.4B	Q4_K_M	18.6 GB	~33 tok/s ModelFit est.
Gemma 2 27B Instruct27B	Q4_K_M	23 GB	~31 tok/s ModelFit est.
Gemma 3 27B Instruct27B	Q4_K_M	23 GB	~32 tok/s ModelFit est.
Qwen3.5 27B Instruct27B	Q8_0	29 GB	~21 tok/s ModelFit est.
Qwen3.6 27B27B	Q8_0	32 GB	~21 tok/s ModelFit est.
Jebadiah 27B27B	Q8_0	31 GB	~21 tok/s ModelFit est.
Mistral Small 3.124B	Q8_0	25 GB	~22 tok/s ModelFit est.
LFM2 24B-A2B Instruct24B MoE	Q4_K_M	15.3 GB	~148 tok/s ModelFit est.
Devstral Small 2 24B24B	Q4_K_M	16.6 GB	~36 tok/s ModelFit est.
Mistral Small 22B22B	Q4_K_M	18.4 GB	~40 tok/s ModelFit est.
Qwen2.5 14B Instruct14B	Q4_K_M	12.3 GB	~55 tok/s ModelFit est.
Phi-3 Medium 14B14B	Q4_K_M	12.3 GB	~40 tok/s our est.
Phi-4 14B14B	Q8_0	15.9 GB	~35 tok/s ModelFit est.
Qwen2.5 Coder 14B14B	Q4_K_M	12.3 GB	~55 tok/s ModelFit est.
Qwen3 14B14B	Q8_0	17.3 GB	~35 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 14B14B	Q8_0	16.0 GB	~35 tok/s ModelFit est.
Ministral 3 14B13.5B	Q4_K_M	10.3 GB	~55 tok/s ModelFit est.
Mellum2.1 12B-A2.5B Thinking12.1B MoE	Q4_K_M	9.3 GB	~263 tok/s our est.
Mistral Nemo 12B12B	Q8_0	13.4 GB	~41 tok/s ModelFit est.
Gemma 3 12B Instruct12B	Q4_K_M	10.7 GB	~63 tok/s ModelFit est.
Gemma 4 12B12B	Q8_0	14.1 GB	~41 tok/s ModelFit est.
GLM-4-9B-Chat9.4B	Q8_0	11.2 GB	~54 tok/s ModelFit est.
Gemma 2 9B Instruct9B	Q4_K_M	8.2 GB	~67 tok/s ModelFit est.
Ornith 1.0 9B9B	Q4_K_M	6.7 GB	~88 tok/s ModelFit est.
MiMo-V2.6-Distill-Qwen-9B9B	Q4_K_M	6.9 GB	~86 tok/s ModelFit est.
Jebadiah 9B9B	Q8_0	11.0 GB	~58 tok/s ModelFit est.
Granite 4.2 8B8.8B	Q4_K_M	6.1 GB	~80 tok/s ModelFit est.
Granite Switch 4.1 8B8.8B	Q4_K_M	7.6 GB	~68 tok/s ModelFit est.
MiniCPM-V 4.5 8B8.7B	Q4_K_M	6.8 GB	~86 tok/s ModelFit est.
Ministral 3 8B8.4B	Q4_K_M	7.1 GB	~76 tok/s ModelFit est.
LFM2.5 8B-A1B8.3B MoE	Q4_K_M	6.6 GB	~172 tok/s ModelFit est.
Llama 3.1 8B Instruct8B	Q8_0	9.2 GB	~59 tok/s ModelFit est.
Qwen3 8B8B	Q8_0	9.3 GB	~58 tok/s ModelFit est.
Granite 4.1 8B Instruct8B	Q4_K_M	6.6 GB	~77 tok/s ModelFit est.
Mistral 7B Instruct7B	Q8_0	8.4 GB	~61 tok/s ModelFit est.
Qwen2.5 Coder 7B7B	Q4_K_M	6.6 GB	~95 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 7B7B	Q8_0	8.7 GB	~64 tok/s ModelFit est.
Gemma 4 E4B4.5B	Q8_0	8.7 GB	~88 tok/s ModelFit est.
Qwen3.5 4B Instruct4B	Q8_0	5.4 GB	~94 tok/s ModelFit est.
NVIDIA Nemotron 3.5 Bolt E4B4B	Q4_K_M	9.1 GB	~106 tok/s ModelFit est.
Phi-3 Mini 3.8B3.8B	Q4_K_M	4.3 GB	~138 tok/s our est.
Phi-4 Mini 3.8B3.8B	Q8_0	4.9 GB	~94 tok/s ModelFit est.
Granite 4.2 3B3.7B	Q4_K_M	3.2 GB	~141 tok/s ModelFit est.
Ministral 3 3B3.4B	Q4_K_M	4.1 GB	~116 tok/s ModelFit est.
Qwen2.5 3B Instruct3B	Q4_K_M	3.6 GB	~154 tok/s ModelFit est.
Llama 3.2 3B Instruct3B	Q4_K_M	3.6 GB	~139 tok/s ModelFit est.
Granite 4.1 3B Instruct3B	Q4_K_M	3.0 GB	~141 tok/s ModelFit est.
Gemma 4 E2B2.3B	Q8_0	5.7 GB	~154 tok/s ModelFit est.
Gemma 2 2B Instruct2B	Q4_K_M	2.8 GB	~151 tok/s ModelFit est.
Qwen3.5 2B Instruct2B	Q4_K_M	2.8 GB	~155 tok/s ModelFit est.
LFM3 1.7B1.73B	Q4_K_M	2.1 GB	~200 tok/s ModelFit est.
Qwen2.5 1.5B Instruct1.5B	Q4_K_M	2.5 GB	~203 tok/s ModelFit est.
Llama 3.2 1B Instruct1B	Q4_K_M	2.0 GB	~215 tok/s ModelFit est.
Gemma 3 1B Instruct1B	Q8_0	2.0 GB	~204 tok/s ModelFit est.
Qwen3.5 0.8B Instruct0.8B	Q4_K_M	1.8 GB	~221 tok/s ModelFit est.
Qwen2.5 0.5B Instruct0.5B	Q4_K_M	1.8 GB	~260 tok/s ModelFit est.
SmolLM2 360M0.36B	Q4_K_M	1.5 GB	~218 tok/s ModelFit est.

Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method

Fits only at 3-bit or lower

Nemotron 3.5 Super VL 120B-A12B.

Too big for this machine

Qwen3.5 122B-A10B Instruct (75 GB), Devstral 2 123B (78 GB), Ornith 1.5 122B (79 GB), Qwen3.8-Flash-Next (93 GB), Laguna S 2.1 (99 GB), Qwen3-235B-A22B (134 GB), DeepSeek-V4-Flash (177 GB), GLM-5.3-Flash (200 GB), Llama 3.1 405B Instruct (250 GB), Llama 4 Maverick (252 GB), Ling-3.1-flash-1019 (368 GB), DeepSeek-R1 671B (390 GB).

Questions

What is the best AI model for the M2 Ultra (96 GB)?

Nemotron 3 Super 120B-A12B at IQ4_XS: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 68 tok/s, our est.).

How much memory can a model use on the M2 Ultra (96 GB)?

About 72 GB of the 96 GB.

How fast is the M2 Ultra (96 GB) for local LLMs?

Decode speed is limited by memory bandwidth: 800 GB/s. A model that reads 10 GB per token runs at under 80 tokens per second.