Graphics cards
Which AI models run on the RTX 4060 (8 GB)?
The RTX 4060 (8 GB) has 8 GB of memory, about 7.0 GB of it usable for a model, and 272 GB/s of memory bandwidth. 32 of the 110 models in our catalog run on it at 4-bit or better.
Data as of 2 November 2026
Models that run on it
| Model | Best build | Memory | Speed |
|---|---|---|---|
| Nemotron Nano 9B v29B | Q4_K_M | 6.6 GB | ~27 tok/s our est. |
| Qwen2.5 7B Instruct7B | Q4_K_M | 6.6 GB | ~47 tok/s ModelFit est. |
| Gemma 3 4B Instruct4B | Q8_0 | 5.0 GB | ~50 tok/s ModelFit est. |
| Ornith 1.0 9B9B | Q4_K_M | 6.7 GB | ~43 tok/s ModelFit est. |
| MiMo-V2.6-Distill-Qwen-9B9B | Q4_K_M | 6.9 GB | ~42 tok/s ModelFit est. |
| Jebadiah 9B9B | Q4_K_M | 6.9 GB | ~42 tok/s ModelFit est. |
| Granite 4.2 8B8.8B | Q4_K_M | 6.1 GB | ~38 tok/s ModelFit est. |
| MiniCPM-V 4.5 8B8.7B | Q4_K_M | 6.8 GB | ~42 tok/s ModelFit est. |
| LFM2.5 8B-A1B8.3B MoE | Q4_K_M | 6.6 GB | ~118 tok/s ModelFit est. |
| Granite 4.1 8B Instruct8B | Q4_K_M | 6.6 GB | ~37 tok/s ModelFit est. |
| Mistral 7B Instruct7B | Q5_K_M | 5.9 GB | ~39 tok/s ModelFit est. |
| Qwen2.5 Coder 7B7B | Q4_K_M | 6.6 GB | ~47 tok/s ModelFit est. |
| DeepSeek-R1 Distill Qwen 7B7B | Q4_K_M | 6.6 GB | ~47 tok/s ModelFit est. |
| Gemma 4 E4B4.5B | Q4_K_M | 5.1 GB | ~61 tok/s ModelFit est. |
| Qwen3.5 4B Instruct4B | Q8_0 | 5.4 GB | ~47 tok/s ModelFit est. |
| Phi-3 Mini 3.8B3.8B | Q4_K_M | 4.3 GB | ~47 tok/s our est. |
| Phi-4 Mini 3.8B3.8B | Q8_0 | 4.9 GB | ~47 tok/s ModelFit est. |
| Granite 4.2 3B3.7B | Q4_K_M | 3.2 GB | ~85 tok/s ModelFit est. |
| Ministral 3 3B3.4B | Q4_K_M | 4.1 GB | ~63 tok/s ModelFit est. |
| Qwen2.5 3B Instruct3B | Q4_K_M | 3.6 GB | ~97 tok/s ModelFit est. |
| Llama 3.2 3B Instruct3B | Q4_K_M | 3.6 GB | ~83 tok/s ModelFit est. |
| Granite 4.1 3B Instruct3B | Q4_K_M | 3.0 GB | ~84 tok/s ModelFit est. |
| Gemma 4 E2B2.3B | Q8_0 | 5.7 GB | ~97 tok/s ModelFit est. |
| Gemma 2 2B Instruct2B | Q4_K_M | 2.8 GB | ~94 tok/s ModelFit est. |
| Qwen3.5 2B Instruct2B | Q4_K_M | 2.8 GB | ~98 tok/s ModelFit est. |
| LFM3 1.7B1.73B | Q4_K_M | 2.1 GB | ~159 tok/s ModelFit est. |
| Qwen2.5 1.5B Instruct1.5B | Q4_K_M | 2.5 GB | ~164 tok/s ModelFit est. |
| Llama 3.2 1B Instruct1B | Q4_K_M | 2.0 GB | ~185 tok/s ModelFit est. |
| Gemma 3 1B Instruct1B | Q8_0 | 2.0 GB | ~165 tok/s ModelFit est. |
| Qwen3.5 0.8B Instruct0.8B | Q4_K_M | 1.8 GB | ~199 tok/s ModelFit est. |
| Qwen2.5 0.5B Instruct0.5B | Q4_K_M | 1.8 GB | ~307 tok/s ModelFit est. |
| SmolLM2 360M0.36B | Q4_K_M | 1.5 GB | ~193 tok/s ModelFit est. |
Speeds come from ModelFit's estimates (CC BY 4.0) where it covers the model, otherwise from our bandwidth formula. Method
Too big for this machine
Ministral 3 8B (7.1 GB), GLM-4-9B-Chat (7.5 GB), Granite Switch 4.1 8B (7.6 GB), Llama 3.1 8B Instruct (7.7 GB), Qwen3 8B (7.7 GB), Gemma 2 9B Instruct (8.2 GB), Qwen3.5 9B Instruct (8.2 GB), NVIDIA Nemotron 3.5 Bolt E4B (9.1 GB), Gemma 4 12B (9.2 GB), Mellum2.1 12B-A2.5B Thinking (9.3 GB), Ministral 3 14B (10.3 GB), Mistral Nemo 12B (10.7 GB) and more.
Questions
What is the best AI model for the RTX 4060 (8 GB)?
Nemotron Nano 9B v2 at Q4_K_M: the largest model on our shortlist of current models that runs at 4-bit or better at a usable speed (about 27 tok/s, our est.).
How much memory can a model use on the RTX 4060 (8 GB)?
About 7.0 GB of the 8 GB.
How fast is the RTX 4060 (8 GB) for local LLMs?
Decode speed is limited by memory bandwidth: 272 GB/s. A model that reads 10 GB per token runs at under 27 tokens per second.