Graphics cards
Which AI models run on the RTX 4080 SUPER (16 GB)?
The RTX 4080 SUPER (16 GB) has 16 GB of memory, about 15.0 GB of it usable for a model, and 736 GB/s of memory bandwidth. 51 of the 110 models in our catalog run on it at 4-bit or better.
Data as of 2 November 2026
Models that run on it
| Model | Best build | Memory | Speed |
|---|---|---|---|
| Nemotron Nano 9B v29B | Q8_0 | 10.8 GB | ~42 tok/s our est. |
| Qwen3.5 9B Instruct9B | Q8_0 | 12.0 GB | ~69 tok/s ModelFit est. |
| Qwen2.5 7B Instruct7B | Q4_K_M | 6.6 GB | ~118 tok/s ModelFit est. |
| Gemma 3 4B Instruct4B | Q8_0 | 5.0 GB | ~124 tok/s ModelFit est. |
| Qwen2.5 14B Instruct14B | Q4_K_M | 12.3 GB | ~63 tok/s ModelFit est. |
| Phi-3 Medium 14B14B | Q4_K_M | 12.3 GB | ~37 tok/s our est. |
| Phi-4 14B14B | Q4_K_M | 12.8 GB | ~62 tok/s ModelFit est. |
| Qwen2.5 Coder 14B14B | Q4_K_M | 12.3 GB | ~63 tok/s ModelFit est. |
| Qwen3 14B14B | Q4_K_M | 12.3 GB | ~63 tok/s ModelFit est. |
| DeepSeek-R1 Distill Qwen 14B14B | Q4_K_M | 12.3 GB | ~63 tok/s ModelFit est. |
| Ministral 3 14B13.5B | Q4_K_M | 10.3 GB | ~63 tok/s ModelFit est. |
| Mellum2.1 12B-A2.5B Thinking12.1B MoE | Q4_K_M | 9.3 GB | ~242 tok/s our est. |
| Mistral Nemo 12B12B | Q8_0 | 13.4 GB | ~47 tok/s ModelFit est. |
| Gemma 3 12B Instruct12B | Q4_K_M | 10.7 GB | ~73 tok/s ModelFit est. |
| Gemma 4 12B12B | Q8_0 | 14.1 GB | ~46 tok/s ModelFit est. |
| GLM-4-9B-Chat9.4B | Q8_0 | 11.2 GB | ~62 tok/s ModelFit est. |
| Gemma 2 9B Instruct9B | Q4_K_M | 8.2 GB | ~80 tok/s ModelFit est. |
| Ornith 1.0 9B9B | Q4_K_M | 6.7 GB | ~109 tok/s ModelFit est. |
| MiMo-V2.6-Distill-Qwen-9B9B | Q4_K_M | 6.9 GB | ~106 tok/s ModelFit est. |
| Jebadiah 9B9B | Q8_0 | 11.0 GB | ~68 tok/s ModelFit est. |
| Granite 4.2 8B8.8B | Q4_K_M | 6.1 GB | ~97 tok/s ModelFit est. |
| Granite Switch 4.1 8B8.8B | Q4_K_M | 7.6 GB | ~81 tok/s ModelFit est. |
| MiniCPM-V 4.5 8B8.7B | Q4_K_M | 6.8 GB | ~105 tok/s ModelFit est. |
| Ministral 3 8B8.4B | Q4_K_M | 7.1 GB | ~91 tok/s ModelFit est. |
| LFM2.5 8B-A1B8.3B MoE | Q4_K_M | 6.6 GB | ~255 tok/s ModelFit est. |
| Llama 3.1 8B Instruct8B | Q8_0 | 9.2 GB | ~69 tok/s ModelFit est. |
| Qwen3 8B8B | Q8_0 | 9.3 GB | ~67 tok/s ModelFit est. |
| Granite 4.1 8B Instruct8B | Q4_K_M | 6.6 GB | ~94 tok/s ModelFit est. |
| Mistral 7B Instruct7B | Q8_0 | 8.4 GB | ~72 tok/s ModelFit est. |
| Qwen2.5 Coder 7B7B | Q4_K_M | 6.6 GB | ~118 tok/s ModelFit est. |
| DeepSeek-R1 Distill Qwen 7B7B | Q8_0 | 8.7 GB | ~75 tok/s ModelFit est. |
| Gemma 4 E4B4.5B | Q8_0 | 8.7 GB | ~109 tok/s ModelFit est. |
| Qwen3.5 4B Instruct4B | Q8_0 | 5.4 GB | ~118 tok/s ModelFit est. |
| NVIDIA Nemotron 3.5 Bolt E4B4B | Q4_K_M | 9.1 GB | ~135 tok/s ModelFit est. |
| Phi-3 Mini 3.8B3.8B | Q4_K_M | 4.3 GB | ~127 tok/s our est. |
| Phi-4 Mini 3.8B3.8B | Q8_0 | 4.9 GB | ~117 tok/s ModelFit est. |
| Granite 4.2 3B3.7B | Q4_K_M | 3.2 GB | ~195 tok/s ModelFit est. |
| Ministral 3 3B3.4B | Q4_K_M | 4.1 GB | ~151 tok/s ModelFit est. |
| Qwen2.5 3B Instruct3B | Q4_K_M | 3.6 GB | ~218 tok/s ModelFit est. |
| Llama 3.2 3B Instruct3B | Q4_K_M | 3.6 GB | ~191 tok/s ModelFit est. |
| Granite 4.1 3B Instruct3B | Q4_K_M | 3.0 GB | ~194 tok/s ModelFit est. |
| Gemma 4 E2B2.3B | Q8_0 | 5.7 GB | ~218 tok/s ModelFit est. |
| Gemma 2 2B Instruct2B | Q4_K_M | 2.8 GB | ~213 tok/s ModelFit est. |
| Qwen3.5 2B Instruct2B | Q4_K_M | 2.8 GB | ~220 tok/s ModelFit est. |
| LFM3 1.7B1.73B | Q4_K_M | 2.1 GB | ~318 tok/s ModelFit est. |
| Qwen2.5 1.5B Instruct1.5B | Q4_K_M | 2.5 GB | ~325 tok/s ModelFit est. |
| Llama 3.2 1B Instruct1B | Q4_K_M | 2.0 GB | ~354 tok/s ModelFit est. |
| Gemma 3 1B Instruct1B | Q8_0 | 2.0 GB | ~327 tok/s ModelFit est. |
| Qwen3.5 0.8B Instruct0.8B | Q4_K_M | 1.8 GB | ~371 tok/s ModelFit est. |
| Qwen2.5 0.5B Instruct0.5B | Q4_K_M | 1.8 GB | ~486 tok/s ModelFit est. |
| SmolLM2 360M0.36B | Q4_K_M | 1.5 GB | ~363 tok/s ModelFit est. |
Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method
Fits only at 3-bit or lower
Too big for this machine
gpt-oss-20b (15.1 GB), LFM2 24B-A2B Instruct (15.3 GB), Mistral Small 3.1 (16.4 GB), Devstral Small 2 24B (16.6 GB), Gemma 4 26B-A4B (17.4 GB), Qwen3.5 27B Instruct (17.4 GB), Qwen3.8-27B (17.9 GB), Granite 4.2 30B (17.9 GB), Jebadiah 27B (18.2 GB), Muse Glimmer 30B (18.3 GB), Mistral Small 22B (18.4 GB), AREX-2 27B (18.6 GB) and more.
Questions
What is the best AI model for the RTX 4080 SUPER (16 GB)?
Qwen3.5 9B Instruct at Q8_0: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 69 tok/s, ModelFit est.).
How much memory can a model use on the RTX 4080 SUPER (16 GB)?
About 15.0 GB of the 16 GB.
How fast is the RTX 4080 SUPER (16 GB) for local LLMs?
Decode speed is limited by memory bandwidth: 736 GB/s. A model that reads 10 GB per token runs at under 74 tokens per second.