Graphics cards
Which AI models run on the RTX 5070 (12 GB)?
The RTX 5070 (12 GB) has 12 GB of memory, about 11.0 GB of it usable for a model, and 672 GB/s of memory bandwidth. 45 of the 110 models in our catalog run on it at 4-bit or better.
Data as of 2 November 2026
Models that run on it
| Model | Best build | Memory | Speed |
|---|---|---|---|
| Nemotron Nano 9B v29B | Q8_0 | 10.8 GB | ~39 tok/s our est. |
| Qwen3.5 9B Instruct9B | Q4_K_M | 8.2 GB | ~104 tok/s ModelFit est. |
| Qwen2.5 7B Instruct7B | Q4_K_M | 6.6 GB | ~113 tok/s ModelFit est. |
| Gemma 3 4B Instruct4B | Q8_0 | 5.0 GB | ~119 tok/s ModelFit est. |
| Ministral 3 14B13.5B | Q4_K_M | 10.3 GB | ~61 tok/s ModelFit est. |
| Mellum2.1 12B-A2.5B Thinking12.1B MoE | Q4_K_M | 9.3 GB | ~221 tok/s our est. |
| Mistral Nemo 12B12B | Q4_K_M | 10.7 GB | ~75 tok/s ModelFit est. |
| Gemma 3 12B Instruct12B | Q4_K_M | 10.7 GB | ~70 tok/s ModelFit est. |
| Gemma 4 12B12B | Q4_K_M | 9.2 GB | ~70 tok/s ModelFit est. |
| GLM-4-9B-Chat9.4B | Q4_K_M | 7.5 GB | ~90 tok/s ModelFit est. |
| Gemma 2 9B Instruct9B | Q4_K_M | 8.2 GB | ~76 tok/s ModelFit est. |
| Ornith 1.0 9B9B | Q4_K_M | 6.7 GB | ~104 tok/s ModelFit est. |
| MiMo-V2.6-Distill-Qwen-9B9B | Q4_K_M | 6.9 GB | ~101 tok/s ModelFit est. |
| Jebadiah 9B9B | Q5_K_M | 7.8 GB | ~91 tok/s ModelFit est. |
| Granite 4.2 8B8.8B | Q4_K_M | 6.1 GB | ~93 tok/s ModelFit est. |
| Granite Switch 4.1 8B8.8B | Q4_K_M | 7.6 GB | ~77 tok/s ModelFit est. |
| MiniCPM-V 4.5 8B8.7B | Q4_K_M | 6.8 GB | ~101 tok/s ModelFit est. |
| Ministral 3 8B8.4B | Q4_K_M | 7.1 GB | ~87 tok/s ModelFit est. |
| LFM2.5 8B-A1B8.3B MoE | Q4_K_M | 6.6 GB | ~246 tok/s ModelFit est. |
| Llama 3.1 8B Instruct8B | Q8_0 | 9.2 GB | ~66 tok/s ModelFit est. |
| Qwen3 8B8B | Q8_0 | 9.3 GB | ~64 tok/s ModelFit est. |
| Granite 4.1 8B Instruct8B | Q4_K_M | 6.6 GB | ~89 tok/s ModelFit est. |
| Mistral 7B Instruct7B | Q8_0 | 8.4 GB | ~69 tok/s ModelFit est. |
| Qwen2.5 Coder 7B7B | Q4_K_M | 6.6 GB | ~113 tok/s ModelFit est. |
| DeepSeek-R1 Distill Qwen 7B7B | Q8_0 | 8.7 GB | ~72 tok/s ModelFit est. |
| Gemma 4 E4B4.5B | Q8_0 | 8.7 GB | ~105 tok/s ModelFit est. |
| Qwen3.5 4B Instruct4B | Q8_0 | 5.4 GB | ~113 tok/s ModelFit est. |
| NVIDIA Nemotron 3.5 Bolt E4B4B | Q4_K_M | 9.1 GB | ~130 tok/s ModelFit est. |
| Phi-3 Mini 3.8B3.8B | Q4_K_M | 4.3 GB | ~116 tok/s our est. |
| Phi-4 Mini 3.8B3.8B | Q8_0 | 4.9 GB | ~112 tok/s ModelFit est. |
| Granite 4.2 3B3.7B | Q4_K_M | 3.2 GB | ~188 tok/s ModelFit est. |
| Ministral 3 3B3.4B | Q4_K_M | 4.1 GB | ~145 tok/s ModelFit est. |
| Qwen2.5 3B Instruct3B | Q4_K_M | 3.6 GB | ~210 tok/s ModelFit est. |
| Llama 3.2 3B Instruct3B | Q4_K_M | 3.6 GB | ~184 tok/s ModelFit est. |
| Granite 4.1 3B Instruct3B | Q4_K_M | 3.0 GB | ~187 tok/s ModelFit est. |
| Gemma 4 E2B2.3B | Q8_0 | 5.7 GB | ~210 tok/s ModelFit est. |
| Gemma 2 2B Instruct2B | Q4_K_M | 2.8 GB | ~205 tok/s ModelFit est. |
| Qwen3.5 2B Instruct2B | Q4_K_M | 2.8 GB | ~212 tok/s ModelFit est. |
| LFM3 1.7B1.73B | Q4_K_M | 2.1 GB | ~309 tok/s ModelFit est. |
| Qwen2.5 1.5B Instruct1.5B | Q4_K_M | 2.5 GB | ~316 tok/s ModelFit est. |
| Llama 3.2 1B Instruct1B | Q4_K_M | 2.0 GB | ~345 tok/s ModelFit est. |
| Gemma 3 1B Instruct1B | Q8_0 | 2.0 GB | ~318 tok/s ModelFit est. |
| Qwen3.5 0.8B Instruct0.8B | Q4_K_M | 1.8 GB | ~362 tok/s ModelFit est. |
| Qwen2.5 0.5B Instruct0.5B | Q4_K_M | 1.8 GB | ~478 tok/s ModelFit est. |
| SmolLM2 360M0.36B | Q4_K_M | 1.5 GB | ~354 tok/s ModelFit est. |
Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method
Too big for this machine
Qwen2.5 14B Instruct (12.3 GB), Phi-3 Medium 14B (12.3 GB), Qwen2.5 Coder 14B (12.3 GB), Qwen3 14B (12.3 GB), DeepSeek-R1 Distill Qwen 14B (12.3 GB), Phi-4 14B (12.8 GB), gpt-oss-20b (15.1 GB), LFM2 24B-A2B Instruct (15.3 GB), Mistral Small 3.1 (16.4 GB), Devstral Small 2 24B (16.6 GB), Gemma 4 26B-A4B (17.4 GB), Qwen3.5 27B Instruct (17.4 GB) and more.
Questions
What is the best AI model for the RTX 5070 (12 GB)?
Qwen3.5 9B Instruct at Q4_K_M: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 104 tok/s, ModelFit est.).
How much memory can a model use on the RTX 5070 (12 GB)?
About 11.0 GB of the 12 GB.
How fast is the RTX 5070 (12 GB) for local LLMs?
Decode speed is limited by memory bandwidth: 672 GB/s. A model that reads 10 GB per token runs at under 67 tokens per second.