Macs

Which AI models run on the M3 Ultra (96 GB, 80-core GPU)?

The M3 Ultra (96 GB, 80-core GPU) has 96 GB of memory, about 72 GB of it usable for a model, and 819 GB/s of memory bandwidth. 97 of the 110 models in our catalog run on it at 4-bit or better.

Data as of 2 November 2026 · Vendor specs

Compare with other machines

96 GBmemory ~72 GB usable for a model

819 GB/sbandwidth, sets the speed

97models run at 4-bit or better

Our pick for this machine Nemotron 3 Super 120B-A12B IQ4_XS · 67 of 72 GB · ~70 tok/s our est. The largest model on our shortlist of current models that runs at 4-bit or better and 10 tok/s or more.

Models that run on it

Model	Best build	Memory	Speed
Nemotron 3 Super 120B-A12B120B MoE	IQ4_XS	67 GB	~70 tok/s our est.
gpt-oss-120b117B MoE	MXFP4	68 GB	~101 tok/s ModelFit est.
Llama 3.3 70B70.6B	Q6_K	60 GB	~11 tok/s ModelFit est.
Llama 3.3 Nemotron Super 49B49B	Q6_K	42 GB	~11 tok/s our est.
Qwen3.6-35B-A3B35B MoE	Q8_0	41 GB	~112 tok/s ModelFit est.
Gemma 4 31B30.7B	Q8_0	33 GB	~18 tok/s ModelFit est.
Qwen3-Coder-30B-A3B30.5B MoE	Q8_0	34 GB	~90 tok/s ModelFit est.
Nemotron 3 Nano 30B-A3B30B MoE	FP16	62 GB	~75 tok/s our est.
Qwen3.8-27B27.3B	Q8_0	29 GB	~22 tok/s ModelFit est.
Gemma 4 26B-A4B25.2B MoE	Q8_0	30 GB	~85 tok/s ModelFit est.
gpt-oss-20b21B MoE	MXFP4	15.1 GB	~117 tok/s ModelFit est.
Nemotron Nano 9B v29B	FP16	19.4 GB	~25 tok/s our est.
Qwen3.5 9B Instruct9B	Q8_0	12.0 GB	~66 tok/s ModelFit est.
Qwen2.5 7B Instruct7B	Q4_K_M	6.6 GB	~107 tok/s ModelFit est.
Gemma 3 4B Instruct4B	Q8_0	5.0 GB	~111 tok/s ModelFit est.
Ling-3.0-flash124B	IQ4_XS	71 GB	no estimate
Llama 4 Scout109B MoE	Q4_K_M	70 GB	~49 tok/s ModelFit est.
Qwen3-Next 80B-A3B80B MoE	Q4_K_M	53 GB	~143 tok/s ModelFit est.
Kolibri 178.1B	Q4_K_M	52 GB	~128 tok/s ModelFit est.
Llama 3.1 70B Instruct70B	Q4_K_M	44 GB	~15 tok/s ModelFit est.
DeepSeek-R1 Distill Llama 70B70B	Q4_K_M	44 GB	~15 tok/s ModelFit est.
Mixtral 8x7B Instruct46.7B MoE	Q4_K_M	32 GB	~47 tok/s ModelFit est.
Flint 40B-A2.5B39.8B MoE	NVFP4	24 GB	~170 tok/s ModelFit est.
Sage-A1.535.95B	Q4_K_M	23 GB	~161 tok/s ModelFit est.
Qwen3.5 35B-A3B Instruct35B MoE	Q8_0	41 GB	~112 tok/s ModelFit est.
Ornith 1.0 35B35B	Q4_K_M	23 GB	~164 tok/s ModelFit est.
Nanbeige4.3 33B33.1B	Q8_0	37 GB	~18 tok/s ModelFit est.
Poolside Laguna XS.233B	Q4_K_M	25 GB	no estimate
Laguna XS 2.133B	Q4_K_M	22 GB	~154 tok/s ModelFit est.
Qwen3 32B32B	Q8_0	37 GB	~18 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 32B32B	Q8_0	37 GB	~18 tok/s ModelFit est.
Olmo 3.1 32B Instruct32B	Q4_K_M	21 GB	~30 tok/s ModelFit est.
Qwen3 30B30B	Q8_0	32 GB	~90 tok/s ModelFit est.
NVIDIA Nemotron Cascade 2 30B-A3B30B MoE	Q6_K	26 GB	~188 tok/s our est.
Cohere North Mini Code30B	Q4_K_M	20 GB	no estimate
Nemotron 3.5 Lightning 30B-A3B30B MoE	Q4_K_M	27 GB	~112 tok/s ModelFit est.
GLM-4.7-Flash 30B-A3B30B MoE	Q4_K_M	20 GB	~123 tok/s ModelFit est.
Granite Switch 4.1 30B30B	Q4_K_M	23 GB	~28 tok/s ModelFit est.
Muse Glimmer 30B29.8B	Q4_K_M	18.3 GB	~37 tok/s ModelFit est.
Granite 4.2 30B29.3B	Q4_K_M	17.9 GB	~33 tok/s ModelFit est.
AREX-2 27B27.4B	Q4_K_M	18.6 GB	~36 tok/s ModelFit est.
Gemma 2 27B Instruct27B	Q4_K_M	23 GB	~33 tok/s ModelFit est.
Gemma 3 27B Instruct27B	Q4_K_M	23 GB	~35 tok/s ModelFit est.
Qwen3.5 27B Instruct27B	Q8_0	29 GB	~22 tok/s ModelFit est.
Qwen3.6 27B27B	Q8_0	32 GB	~22 tok/s ModelFit est.
Jebadiah 27B27B	Q8_0	31 GB	~22 tok/s ModelFit est.
Mistral Small 3.124B	Q8_0	25 GB	~24 tok/s ModelFit est.
LFM2 24B-A2B Instruct24B MoE	Q4_K_M	15.3 GB	~172 tok/s ModelFit est.
Devstral Small 2 24B24B	Q4_K_M	16.6 GB	~39 tok/s ModelFit est.
Mistral Small 22B22B	Q4_K_M	18.4 GB	~44 tok/s ModelFit est.
Qwen2.5 14B Instruct14B	Q4_K_M	12.3 GB	~60 tok/s ModelFit est.
Phi-3 Medium 14B14B	Q4_K_M	12.3 GB	~41 tok/s our est.
Phi-4 14B14B	Q8_0	15.9 GB	~38 tok/s ModelFit est.
Qwen2.5 Coder 14B14B	Q4_K_M	12.3 GB	~60 tok/s ModelFit est.
Qwen3 14B14B	Q8_0	17.3 GB	~38 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 14B14B	Q8_0	16.0 GB	~38 tok/s ModelFit est.
Ministral 3 14B13.5B	Q4_K_M	10.3 GB	~60 tok/s ModelFit est.
Mellum2.1 12B-A2.5B Thinking12.1B MoE	Q4_K_M	9.3 GB	~269 tok/s our est.
Mistral Nemo 12B12B	Q8_0	13.4 GB	~45 tok/s ModelFit est.
Gemma 3 12B Instruct12B	Q4_K_M	10.7 GB	~69 tok/s ModelFit est.
Gemma 4 12B12B	Q8_0	14.1 GB	~44 tok/s ModelFit est.
GLM-4-9B-Chat9.4B	Q8_0	11.2 GB	~60 tok/s ModelFit est.
Gemma 2 9B Instruct9B	Q4_K_M	8.2 GB	~75 tok/s ModelFit est.
Ornith 1.0 9B9B	Q4_K_M	6.7 GB	~99 tok/s ModelFit est.
MiMo-V2.6-Distill-Qwen-9B9B	Q4_K_M	6.9 GB	~96 tok/s ModelFit est.
Jebadiah 9B9B	Q8_0	11.0 GB	~64 tok/s ModelFit est.
Granite 4.2 8B8.8B	Q4_K_M	6.1 GB	~89 tok/s ModelFit est.
Granite Switch 4.1 8B8.8B	Q4_K_M	7.6 GB	~76 tok/s ModelFit est.
MiniCPM-V 4.5 8B8.7B	Q4_K_M	6.8 GB	~96 tok/s ModelFit est.
Ministral 3 8B8.4B	Q4_K_M	7.1 GB	~84 tok/s ModelFit est.
LFM2.5 8B-A1B8.3B MoE	Q4_K_M	6.6 GB	~203 tok/s ModelFit est.
Llama 3.1 8B Instruct8B	Q8_0	9.2 GB	~65 tok/s ModelFit est.
Qwen3 8B8B	Q8_0	9.3 GB	~64 tok/s ModelFit est.
Granite 4.1 8B Instruct8B	Q4_K_M	6.6 GB	~86 tok/s ModelFit est.
Mistral 7B Instruct7B	Q8_0	8.4 GB	~68 tok/s ModelFit est.
Qwen2.5 Coder 7B7B	Q4_K_M	6.6 GB	~107 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 7B7B	Q8_0	8.7 GB	~71 tok/s ModelFit est.
Gemma 4 E4B4.5B	Q8_0	8.7 GB	~99 tok/s ModelFit est.
Qwen3.5 4B Instruct4B	Q8_0	5.4 GB	~106 tok/s ModelFit est.
NVIDIA Nemotron 3.5 Bolt E4B4B	Q4_K_M	9.1 GB	~120 tok/s ModelFit est.
Phi-3 Mini 3.8B3.8B	Q4_K_M	4.3 GB	~141 tok/s our est.
Phi-4 Mini 3.8B3.8B	Q8_0	4.9 GB	~106 tok/s ModelFit est.
Granite 4.2 3B3.7B	Q4_K_M	3.2 GB	~164 tok/s ModelFit est.
Ministral 3 3B3.4B	Q4_K_M	4.1 GB	~132 tok/s ModelFit est.
Qwen2.5 3B Instruct3B	Q4_K_M	3.6 GB	~180 tok/s ModelFit est.
Llama 3.2 3B Instruct3B	Q4_K_M	3.6 GB	~161 tok/s ModelFit est.
Granite 4.1 3B Instruct3B	Q4_K_M	3.0 GB	~164 tok/s ModelFit est.
Gemma 4 E2B2.3B	Q8_0	5.7 GB	~180 tok/s ModelFit est.
Gemma 2 2B Instruct2B	Q4_K_M	2.8 GB	~176 tok/s ModelFit est.
Qwen3.5 2B Instruct2B	Q4_K_M	2.8 GB	~181 tok/s ModelFit est.
LFM3 1.7B1.73B	Q4_K_M	2.1 GB	~241 tok/s ModelFit est.
Qwen2.5 1.5B Instruct1.5B	Q4_K_M	2.5 GB	~245 tok/s ModelFit est.
Llama 3.2 1B Instruct1B	Q4_K_M	2.0 GB	~260 tok/s ModelFit est.
Gemma 3 1B Instruct1B	Q8_0	2.0 GB	~246 tok/s ModelFit est.
Qwen3.5 0.8B Instruct0.8B	Q4_K_M	1.8 GB	~269 tok/s ModelFit est.
Qwen2.5 0.5B Instruct0.5B	Q4_K_M	1.8 GB	~324 tok/s ModelFit est.
SmolLM2 360M0.36B	Q4_K_M	1.5 GB	~265 tok/s ModelFit est.

Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method

Fits only at 3-bit or lower

Nemotron 3.5 Super VL 120B-A12B.

Too big for this machine

Qwen3.5 122B-A10B Instruct (75 GB), Devstral 2 123B (78 GB), Ornith 1.5 122B (79 GB), Qwen3.8-Flash-Next (93 GB), Laguna S 2.1 (99 GB), Qwen3-235B-A22B (134 GB), DeepSeek-V4-Flash (177 GB), GLM-5.3-Flash (200 GB), Llama 3.1 405B Instruct (250 GB), Llama 4 Maverick (252 GB), Ling-3.1-flash-1019 (368 GB), DeepSeek-R1 671B (390 GB).

Questions

What is the best AI model for the M3 Ultra (96 GB, 80-core GPU)?

Nemotron 3 Super 120B-A12B at IQ4_XS: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 70 tok/s, our est.).

How much memory can a model use on the M3 Ultra (96 GB, 80-core GPU)?

About 72 GB of the 96 GB.

How fast is the M3 Ultra (96 GB, 80-core GPU) for local LLMs?

Decode speed is limited by memory bandwidth: 819 GB/s. A model that reads 10 GB per token runs at under 82 tokens per second.