Graphics cards

Which AI models run on the RTX PRO 6000 Blackwell (96 GB)?

The RTX PRO 6000 Blackwell (96 GB) has 96 GB of memory, about 95 GB of it usable for a model, and 1792 GB/s of memory bandwidth. 101 of the 110 models in our catalog run on it at 4-bit or better.

Data as of 2 November 2026 · Vendor specs

Compare with other machines

96 GBmemory ~95 GB usable for a model

1792 GB/sbandwidth, sets the speed

101models run at 4-bit or better

Our pick for this machine Nemotron 3.5 Super VL 120B-A12B Q4_K_M · 76 of 95 GB · ~135 tok/s our est. The largest model on our shortlist of current models that runs at 4-bit or better and 10 tok/s or more.

Models that run on it

Model	Best build	Memory	Speed
Nemotron 3 Super 120B-A12B120B MoE	Q4_K_M	90 GB	~114 tok/s our est.
Nemotron 3.5 Super VL 120B-A12B120B MoE	Q4_K_M	76 GB	~135 tok/s our est.
gpt-oss-120b117B MoE	MXFP4	68 GB	~229 tok/s ModelFit est.
Llama 3.3 70B70.6B	Q8_0	78 GB	~21 tok/s ModelFit est.
Llama 3.3 Nemotron Super 49B49B	Q8_0	54 GB	~19 tok/s our est.
Qwen3.6-35B-A3B35B MoE	Q8_0	41 GB	~251 tok/s ModelFit est.
Gemma 4 31B30.7B	Q8_0	33 GB	~45 tok/s ModelFit est.
Qwen3-Coder-30B-A3B30.5B MoE	Q8_0	34 GB	~205 tok/s ModelFit est.
Nemotron 3 Nano 30B-A3B30B MoE	FP16	62 GB	~164 tok/s our est.
Qwen3.8-27B27.3B	Q8_0	29 GB	~55 tok/s ModelFit est.
Gemma 4 26B-A4B25.2B MoE	Q8_0	30 GB	~195 tok/s ModelFit est.
gpt-oss-20b21B MoE	MXFP4	15.1 GB	~261 tok/s ModelFit est.
Nemotron Nano 9B v29B	FP16	19.4 GB	~55 tok/s our est.
Qwen3.5 9B Instruct9B	Q8_0	12.0 GB	~153 tok/s ModelFit est.
Qwen2.5 7B Instruct7B	Q4_K_M	6.6 GB	~240 tok/s ModelFit est.
Gemma 3 4B Instruct4B	Q8_0	5.0 GB	~249 tok/s ModelFit est.
Ornith 1.5 122B125.09B	Q4_K_M	79 GB	~170 tok/s ModelFit est.
Ling-3.0-flash124B	Q4_K_M	81 GB	~238 tok/s ModelFit est.
Devstral 2 123B123B	Q4_K_M	78 GB	~21 tok/s ModelFit est.
Qwen3.5 122B-A10B Instruct122B MoE	Q4_K_M	75 GB	~161 tok/s ModelFit est.
Llama 4 Scout109B MoE	Q4_K_M	70 GB	~116 tok/s ModelFit est.
Qwen3-Next 80B-A3B80B MoE	Q8_0	88 GB	~311 tok/s ModelFit est.
Kolibri 178.1B	Q4_K_M	52 GB	~282 tok/s ModelFit est.
Llama 3.1 70B Instruct70B	Q4_K_M	44 GB	~36 tok/s ModelFit est.
DeepSeek-R1 Distill Llama 70B70B	Q4_K_M	44 GB	~36 tok/s ModelFit est.
Mixtral 8x7B Instruct46.7B MoE	Q4_K_M	32 GB	~111 tok/s ModelFit est.
Flint 40B-A2.5B39.8B MoE	NVFP4	24 GB	~363 tok/s ModelFit est.
Sage-A1.535.95B	Q4_K_M	23 GB	~346 tok/s ModelFit est.
Qwen3.5 35B-A3B Instruct35B MoE	Q8_0	41 GB	~251 tok/s ModelFit est.
Ornith 1.0 35B35B	Q4_K_M	23 GB	~350 tok/s ModelFit est.
Nanbeige4.3 33B33.1B	Q8_0	37 GB	~44 tok/s ModelFit est.
Poolside Laguna XS.233B	Q4_K_M	25 GB	no estimate
Laguna XS 2.133B	Q4_K_M	22 GB	~332 tok/s ModelFit est.
Qwen3 32B32B	Q8_0	37 GB	~44 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 32B32B	Q8_0	37 GB	~44 tok/s ModelFit est.
Olmo 3.1 32B Instruct32B	Q4_K_M	21 GB	~73 tok/s ModelFit est.
Qwen3 30B30B	Q8_0	32 GB	~205 tok/s ModelFit est.
NVIDIA Nemotron Cascade 2 30B-A3B30B MoE	Q6_K	26 GB	~411 tok/s our est.
Cohere North Mini Code30B	Q4_K_M	20 GB	no estimate
Nemotron 3.5 Lightning 30B-A3B30B MoE	Q4_K_M	27 GB	~250 tok/s ModelFit est.
GLM-4.7-Flash 30B-A3B30B MoE	Q4_K_M	20 GB	~273 tok/s ModelFit est.
Granite Switch 4.1 30B30B	Q4_K_M	23 GB	~67 tok/s ModelFit est.
Muse Glimmer 30B29.8B	Q4_K_M	18.3 GB	~90 tok/s ModelFit est.
Granite 4.2 30B29.3B	Q4_K_M	17.9 GB	~79 tok/s ModelFit est.
AREX-2 27B27.4B	Q4_K_M	18.6 GB	~87 tok/s ModelFit est.
Gemma 2 27B Instruct27B	Q4_K_M	23 GB	~80 tok/s ModelFit est.
Gemma 3 27B Instruct27B	Q4_K_M	23 GB	~84 tok/s ModelFit est.
Qwen3.5 27B Instruct27B	Q8_0	29 GB	~55 tok/s ModelFit est.
Qwen3.6 27B27B	Q8_0	32 GB	~55 tok/s ModelFit est.
Jebadiah 27B27B	Q8_0	31 GB	~55 tok/s ModelFit est.
Mistral Small 3.124B	Q8_0	25 GB	~58 tok/s ModelFit est.
LFM2 24B-A2B Instruct24B MoE	Q4_K_M	15.3 GB	~366 tok/s ModelFit est.
Devstral Small 2 24B24B	Q4_K_M	16.6 GB	~93 tok/s ModelFit est.
Mistral Small 22B22B	Q4_K_M	18.4 GB	~105 tok/s ModelFit est.
Qwen2.5 14B Instruct14B	Q4_K_M	12.3 GB	~142 tok/s ModelFit est.
Phi-3 Medium 14B14B	Q4_K_M	12.3 GB	~90 tok/s our est.
Phi-4 14B14B	Q8_0	15.9 GB	~91 tok/s ModelFit est.
Qwen2.5 Coder 14B14B	Q4_K_M	12.3 GB	~142 tok/s ModelFit est.
Qwen3 14B14B	Q8_0	17.3 GB	~91 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 14B14B	Q8_0	16.0 GB	~92 tok/s ModelFit est.
Ministral 3 14B13.5B	Q4_K_M	10.3 GB	~142 tok/s ModelFit est.
Mellum2.1 12B-A2.5B Thinking12.1B MoE	Q4_K_M	9.3 GB	~589 tok/s our est.
Mistral Nemo 12B12B	Q8_0	13.4 GB	~108 tok/s ModelFit est.
Gemma 3 12B Instruct12B	Q4_K_M	10.7 GB	~161 tok/s ModelFit est.
Gemma 4 12B12B	Q8_0	14.1 GB	~106 tok/s ModelFit est.
GLM-4-9B-Chat9.4B	Q8_0	11.2 GB	~140 tok/s ModelFit est.
Gemma 2 9B Instruct9B	Q4_K_M	8.2 GB	~173 tok/s ModelFit est.
Ornith 1.0 9B9B	Q4_K_M	6.7 GB	~225 tok/s ModelFit est.
MiMo-V2.6-Distill-Qwen-9B9B	Q4_K_M	6.9 GB	~219 tok/s ModelFit est.
Jebadiah 9B9B	Q8_0	11.0 GB	~150 tok/s ModelFit est.
Granite 4.2 8B8.8B	Q4_K_M	6.1 GB	~204 tok/s ModelFit est.
Granite Switch 4.1 8B8.8B	Q4_K_M	7.6 GB	~175 tok/s ModelFit est.
MiniCPM-V 4.5 8B8.7B	Q4_K_M	6.8 GB	~219 tok/s ModelFit est.
Ministral 3 8B8.4B	Q4_K_M	7.1 GB	~194 tok/s ModelFit est.
LFM2.5 8B-A1B8.3B MoE	Q4_K_M	6.6 GB	~421 tok/s ModelFit est.
Llama 3.1 8B Instruct8B	Q8_0	9.2 GB	~152 tok/s ModelFit est.
Qwen3 8B8B	Q8_0	9.3 GB	~149 tok/s ModelFit est.
Granite 4.1 8B Instruct8B	Q4_K_M	6.6 GB	~198 tok/s ModelFit est.
Mistral 7B Instruct7B	Q8_0	8.4 GB	~159 tok/s ModelFit est.
Qwen2.5 Coder 7B7B	Q4_K_M	6.6 GB	~240 tok/s ModelFit est.
DeepSeek-R1 Distill Qwen 7B7B	Q8_0	8.7 GB	~164 tok/s ModelFit est.
Gemma 4 E4B4.5B	Q8_0	8.7 GB	~225 tok/s ModelFit est.
Qwen3.5 4B Instruct4B	Q8_0	5.4 GB	~239 tok/s ModelFit est.
NVIDIA Nemotron 3.5 Bolt E4B4B	Q4_K_M	9.1 GB	~267 tok/s ModelFit est.
Phi-3 Mini 3.8B3.8B	Q4_K_M	4.3 GB	~308 tok/s our est.
Phi-4 Mini 3.8B3.8B	Q8_0	4.9 GB	~239 tok/s ModelFit est.
Granite 4.2 3B3.7B	Q4_K_M	3.2 GB	~351 tok/s ModelFit est.
Ministral 3 3B3.4B	Q4_K_M	4.1 GB	~291 tok/s ModelFit est.
Qwen2.5 3B Instruct3B	Q4_K_M	3.6 GB	~380 tok/s ModelFit est.
Llama 3.2 3B Instruct3B	Q4_K_M	3.6 GB	~346 tok/s ModelFit est.
Granite 4.1 3B Instruct3B	Q4_K_M	3.0 GB	~350 tok/s ModelFit est.
Gemma 4 E2B2.3B	Q8_0	5.7 GB	~380 tok/s ModelFit est.
Gemma 2 2B Instruct2B	Q4_K_M	2.8 GB	~373 tok/s ModelFit est.
Qwen3.5 2B Instruct2B	Q4_K_M	2.8 GB	~382 tok/s ModelFit est.
LFM3 1.7B1.73B	Q4_K_M	2.1 GB	~484 tok/s ModelFit est.
Qwen2.5 1.5B Instruct1.5B	Q4_K_M	2.5 GB	~491 tok/s ModelFit est.
Llama 3.2 1B Instruct1B	Q4_K_M	2.0 GB	~516 tok/s ModelFit est.
Gemma 3 1B Instruct1B	Q8_0	2.0 GB	~492 tok/s ModelFit est.
Qwen3.5 0.8B Instruct0.8B	Q4_K_M	1.8 GB	~529 tok/s ModelFit est.
Qwen2.5 0.5B Instruct0.5B	Q4_K_M	1.8 GB	~611 tok/s ModelFit est.
SmolLM2 360M0.36B	Q4_K_M	1.5 GB	~523 tok/s ModelFit est.

Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method

Fits only at 3-bit or lower

Qwen3.8-Flash-Next.

Too big for this machine

Laguna S 2.1 (99 GB), Qwen3-235B-A22B (134 GB), DeepSeek-V4-Flash (177 GB), GLM-5.3-Flash (200 GB), Llama 3.1 405B Instruct (250 GB), Llama 4 Maverick (252 GB), Ling-3.1-flash-1019 (368 GB), DeepSeek-R1 671B (390 GB).

Questions

What is the best AI model for the RTX PRO 6000 Blackwell (96 GB)?

Nemotron 3.5 Super VL 120B-A12B at Q4_K_M: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 135 tok/s, our est.).

How much memory can a model use on the RTX PRO 6000 Blackwell (96 GB)?

About 95 GB of the 96 GB.

How fast is the RTX PRO 6000 Blackwell (96 GB) for local LLMs?

Decode speed is limited by memory bandwidth: 1792 GB/s. A model that reads 10 GB per token runs at under 179 tokens per second.