Graphics cards
Which AI models run on the RTX 4070 (12 GB)?
The RTX 4070 (12 GB) has 12 GB of memory, about 11.0 GB of it usable for a model, and 504 GB/s of memory bandwidth. 45 of the 110 models in our catalog run on it at 4-bit or better.
Data as of 2 November 2026
Models that run on it
| Model | Best build | Memory | Speed |
|---|---|---|---|
| Nemotron Nano 9B v29B | Q8_0 | 10.8 GB | ~29 tok/s our est. |
| Qwen3.5 9B Instruct9B | Q4_K_M | 8.2 GB | ~78 tok/s ModelFit est. |
| Qwen2.5 7B Instruct7B | Q4_K_M | 6.6 GB | ~85 tok/s ModelFit est. |
| Gemma 3 4B Instruct4B | Q8_0 | 5.0 GB | ~90 tok/s ModelFit est. |
| Ministral 3 14B13.5B | Q4_K_M | 10.3 GB | ~45 tok/s ModelFit est. |
| Mellum2.1 12B-A2.5B Thinking12.1B MoE | Q4_K_M | 9.3 GB | ~166 tok/s our est. |
| Mistral Nemo 12B12B | Q4_K_M | 10.7 GB | ~56 tok/s ModelFit est. |
| Gemma 3 12B Instruct12B | Q4_K_M | 10.7 GB | ~52 tok/s ModelFit est. |
| Gemma 4 12B12B | Q4_K_M | 9.2 GB | ~52 tok/s ModelFit est. |
| GLM-4-9B-Chat9.4B | Q4_K_M | 7.5 GB | ~67 tok/s ModelFit est. |
| Gemma 2 9B Instruct9B | Q4_K_M | 8.2 GB | ~57 tok/s ModelFit est. |
| Ornith 1.0 9B9B | Q4_K_M | 6.7 GB | ~78 tok/s ModelFit est. |
| MiMo-V2.6-Distill-Qwen-9B9B | Q4_K_M | 6.9 GB | ~76 tok/s ModelFit est. |
| Jebadiah 9B9B | Q5_K_M | 7.8 GB | ~68 tok/s ModelFit est. |
| Granite 4.2 8B8.8B | Q4_K_M | 6.1 GB | ~69 tok/s ModelFit est. |
| Granite Switch 4.1 8B8.8B | Q4_K_M | 7.6 GB | ~57 tok/s ModelFit est. |
| MiniCPM-V 4.5 8B8.7B | Q4_K_M | 6.8 GB | ~76 tok/s ModelFit est. |
| Ministral 3 8B8.4B | Q4_K_M | 7.1 GB | ~65 tok/s ModelFit est. |
| LFM2.5 8B-A1B8.3B MoE | Q4_K_M | 6.6 GB | ~196 tok/s ModelFit est. |
| Llama 3.1 8B Instruct8B | Q8_0 | 9.2 GB | ~49 tok/s ModelFit est. |
| Qwen3 8B8B | Q8_0 | 9.3 GB | ~47 tok/s ModelFit est. |
| Granite 4.1 8B Instruct8B | Q4_K_M | 6.6 GB | ~67 tok/s ModelFit est. |
| Mistral 7B Instruct7B | Q8_0 | 8.4 GB | ~51 tok/s ModelFit est. |
| Qwen2.5 Coder 7B7B | Q4_K_M | 6.6 GB | ~85 tok/s ModelFit est. |
| DeepSeek-R1 Distill Qwen 7B7B | Q8_0 | 8.7 GB | ~53 tok/s ModelFit est. |
| Gemma 4 E4B4.5B | Q8_0 | 8.7 GB | ~79 tok/s ModelFit est. |
| Qwen3.5 4B Instruct4B | Q8_0 | 5.4 GB | ~85 tok/s ModelFit est. |
| NVIDIA Nemotron 3.5 Bolt E4B4B | Q4_K_M | 9.1 GB | ~98 tok/s ModelFit est. |
| Phi-3 Mini 3.8B3.8B | Q4_K_M | 4.3 GB | ~87 tok/s our est. |
| Phi-4 Mini 3.8B3.8B | Q8_0 | 4.9 GB | ~85 tok/s ModelFit est. |
| Granite 4.2 3B3.7B | Q4_K_M | 3.2 GB | ~146 tok/s ModelFit est. |
| Ministral 3 3B3.4B | Q4_K_M | 4.1 GB | ~111 tok/s ModelFit est. |
| Qwen2.5 3B Instruct3B | Q4_K_M | 3.6 GB | ~165 tok/s ModelFit est. |
| Llama 3.2 3B Instruct3B | Q4_K_M | 3.6 GB | ~143 tok/s ModelFit est. |
| Granite 4.1 3B Instruct3B | Q4_K_M | 3.0 GB | ~145 tok/s ModelFit est. |
| Gemma 4 E2B2.3B | Q8_0 | 5.7 GB | ~165 tok/s ModelFit est. |
| Gemma 2 2B Instruct2B | Q4_K_M | 2.8 GB | ~160 tok/s ModelFit est. |
| Qwen3.5 2B Instruct2B | Q4_K_M | 2.8 GB | ~166 tok/s ModelFit est. |
| LFM3 1.7B1.73B | Q4_K_M | 2.1 GB | ~252 tok/s ModelFit est. |
| Qwen2.5 1.5B Instruct1.5B | Q4_K_M | 2.5 GB | ~259 tok/s ModelFit est. |
| Llama 3.2 1B Instruct1B | Q4_K_M | 2.0 GB | ~286 tok/s ModelFit est. |
| Gemma 3 1B Instruct1B | Q8_0 | 2.0 GB | ~260 tok/s ModelFit est. |
| Qwen3.5 0.8B Instruct0.8B | Q4_K_M | 1.8 GB | ~302 tok/s ModelFit est. |
| Qwen2.5 0.5B Instruct0.5B | Q4_K_M | 1.8 GB | ~420 tok/s ModelFit est. |
| SmolLM2 360M0.36B | Q4_K_M | 1.5 GB | ~295 tok/s ModelFit est. |
Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method
Too big for this machine
Qwen2.5 14B Instruct (12.3 GB), Phi-3 Medium 14B (12.3 GB), Qwen2.5 Coder 14B (12.3 GB), Qwen3 14B (12.3 GB), DeepSeek-R1 Distill Qwen 14B (12.3 GB), Phi-4 14B (12.8 GB), gpt-oss-20b (15.1 GB), LFM2 24B-A2B Instruct (15.3 GB), Mistral Small 3.1 (16.4 GB), Devstral Small 2 24B (16.6 GB), Gemma 4 26B-A4B (17.4 GB), Qwen3.5 27B Instruct (17.4 GB) and more.
Questions
What is the best AI model for the RTX 4070 (12 GB)?
Qwen3.5 9B Instruct at Q4_K_M: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 78 tok/s, ModelFit est.).
How much memory can a model use on the RTX 4070 (12 GB)?
About 11.0 GB of the 12 GB.
How fast is the RTX 4070 (12 GB) for local LLMs?
Decode speed is limited by memory bandwidth: 504 GB/s. A model that reads 10 GB per token runs at under 50 tokens per second.