FitLocally by GPUwerk

Mistral · 12B

Can you run Mistral Nemo 12B locally?

Mistral Nemo 12B is a 12B-parameter model. At 4-bit or better it runs on 102 of the 120 hardware configurations listed here; the smallest is the RTX 5070 (12 GB). On a 128 GB DGX Spark it runs at Q4_K_M.

Data as of 2 November 2026

Fastest machines for Mistral Nemo 12B

Every machine that runs it, fastest first. Filter by type; your machine is marked. Compare machines running it side by side

  1. 1RTX 5090Graphics card · 32 GB~108 tok/s ModelFit est.Q8_0 · 13.4 of 31 GB
  2. 2RTX PRO 6000 BlackwellGraphics card · 96 GB~108 tok/s ModelFit est.Q8_0 · 13.4 of 95 GB
  3. 32x RTX 5090Graphics card · 64 GB~104 tok/s ModelFit est.Q8_0 · 13.4 of 62 GB
  4. 42x RTX PRO 6000 BlackwellGraphics card · 192 GB~104 tok/s ModelFit est.Q8_0 · 13.4 of 190 GB
  5. 5RTX 5070Graphics card · 12 GB~75 tok/s ModelFit est.Q4_K_M · 10.7 of 11.0 GB
  6. 6M5 UltraMac · 96–512 GB · 5 variants~67 tok/s ModelFit est.Q8_0 · 13.4 GB
  7. 7RTX 4090Graphics card · 24 GB~62 tok/s ModelFit est.Q8_0 · 13.4 of 23 GB
  8. 8RTX 5080Graphics card · 16 GB~62 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  9. 92x RTX 4090Graphics card · 48 GB~61 tok/s ModelFit est.Q8_0 · 13.4 of 46 GB
  10. 10RTX 3090 TiGraphics card · 24 GB~59 tok/s ModelFit est.Q8_0 · 13.4 of 23 GB
  11. 11RTX 5070 TiGraphics card · 16 GB~58 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  12. 122x RTX 3090 TiGraphics card · 48 GB~58 tok/s ModelFit est.Q8_0 · 13.4 of 46 GB
  13. 13RTX 4070 TiGraphics card · 12 GB~56 tok/s ModelFit est.Q4_K_M · 10.7 of 11.0 GB
  14. 14RTX 4070 SUPERGraphics card · 12 GB~56 tok/s ModelFit est.Q4_K_M · 10.7 of 11.0 GB
  15. 15RTX 4070Graphics card · 12 GB~56 tok/s ModelFit est.Q4_K_M · 10.7 of 11.0 GB
  16. 16RTX 3090Graphics card · 24 GB~55 tok/s ModelFit est.Q8_0 · 13.4 of 23 GB
  17. 172x RTX 3090Graphics card · 48 GB~54 tok/s ModelFit est.Q8_0 · 13.4 of 46 GB
  18. 18RX 7900 XTXGraphics card · 24 GB~53 tok/s ModelFit est.Q8_0 · 13.4 of 23 GB
  19. 192x RX 7900 XTXGraphics card · 48 GB~52 tok/s ModelFit est.Q8_0 · 13.4 of 46 GB
  20. 20RTX 4080 SUPERGraphics card · 16 GB~47 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  21. 21RTX 4080Graphics card · 16 GB~46 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  22. 22M3 UltraMac · 96–512 GB · 5 variants~45 tok/s ModelFit est.Q8_0 · 13.4 GB
  23. 23RX 7900 XTGraphics card · 20 GB~45 tok/s ModelFit est.Q8_0 · 13.4 of 19.0 GB
  24. 24RTX 4070 Ti SUPERGraphics card · 16 GB~43 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  25. 25M2 UltraMac · 64–192 GB · 7 variants~41 tok/s ModelFit est.Q8_0 · 13.4 GB
  26. 26RX 9070 XTGraphics card · 16 GB~39 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  27. 27RTX 3060Graphics card · 12 GB~38 tok/s ModelFit est.Q4_K_M · 10.7 of 11.0 GB
  28. 28M5 MaxMac · 48–128 GB · 3 variants~37 tok/s ModelFit est.Q8_0 · 13.4 GB
  29. 29M1 UltraMac · 64–128 GB · 4 variants~37 tok/s ModelFit est.Q8_0 · 13.4 GB
  30. 30M4 MaxMac · 48–128 GB · 3 variants~31 tok/s ModelFit est.Q8_0 · 13.4 GB
  31. 31RTX 5060 TiGraphics card · 16 GB~30 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  32. 32M5 Max · 36 GB · 32cMac · 36 GB~28 tok/s ModelFit est.Q8_0 · 13.4 of 24 GB
  33. 33M3 MaxMac · 48–128 GB · 3 variants~24 tok/s ModelFit est.Q8_0 · 13.4 GB
  34. 34M4 Max · 36 GB · 32cMac · 36 GB~24 tok/s ModelFit est.Q8_0 · 13.4 of 24 GB
  35. 35M2 MaxMac · 32–96 GB · 5 variants~23 tok/s ModelFit est.Q8_0 · 13.4 GB
  36. 36RX 9060 XTGraphics card · 16 GB~21 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  37. 37M5 ProMac · 24–64 GB · 6 variants~21 tok/s ModelFit est.Q8_0 · 13.4 GB
  38. 38M1 MaxMac · 32–64 GB · 4 variants~21 tok/s ModelFit est.Q8_0 · 13.4 GB
  39. 39Tesla P100Graphics card · 16 GB~19 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  40. 40RTX 4060 TiGraphics card · 16 GB~18 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  41. 41M3 MaxMac · 36–96 GB · 2 variants~18 tok/s ModelFit est.Q8_0 · 13.4 GB
  42. 42M3 Pro · 18 GBMac · 18 GB~18 tok/s ModelFit est.Q4_K_M · 10.7 of 12.0 GB
  43. 43M4 ProMac · 24–64 GB · 6 variants~17 tok/s ModelFit est.Q8_0 · 13.4 GB
  44. 44Tesla T4Graphics card · 16 GB~16 tok/s ModelFit est.Q8_0 · 13.4 of 15.0 GB
  45. 45Ryzen AI Max+ 395 · 128 GBAI mini PC · 128 GB~14 tok/s ModelFit est.Q8_0 · 13.4 of 96 GB
  46. 46M4Mac · 24–32 GB · 2 variants~13 tok/s ModelFit est.Q4_K_M · 10.7 GB
  47. 47M2 Pro · 32 GBMac · 32 GB~13 tok/s ModelFit est.Q8_0 · 13.4 of 21 GB
  48. 48M6Mac · 24–32 GB · 3 variants~12 tok/s ModelFit est.Q8_0 · 13.4 GB
  49. 49M3 · 24 GBMac · 24 GB~12 tok/s ModelFit est.Q4_K_M · 10.7 of 16.0 GB
  50. 50M1 Pro · 32 GBMac · 32 GB~11 tok/s ModelFit est.Q8_0 · 13.4 of 21 GB
  51. 51M2 · 24 GBMac · 24 GB~11 tok/s ModelFit est.Q4_K_M · 10.7 of 16.0 GB
  52. 52M5Mac · 24–32 GB · 4 variants~10 tok/s ModelFit est.Q8_0 · 13.4 GB
  53. 53M3 Pro · 36 GBMac · 36 GB~10 tok/s ModelFit est.Q8_0 · 13.4 of 24 GB
  54. 54Ryzen AI Max+ 395 · 64 GBAI mini PC · 64 GBno estimateQ4_K_M · 10.7 of 48 GB
  55. 55DGX Spark (GB10)AI mini PC · 128 GBno estimateQ4_K_M · 10.7 of 122 GB
Too big for 18 machines

M2 Pro · 16 GB, M1 Pro · 16 GB, M6 · 16 GB · 12c, M6 · 16 GB · 10c, M5 · 16 GB · 10c, M5 · 16 GB · 8c, M4 · 16 GB, M3 · 16 GB, M2 · 16 GB, M1 · 16 GB, RTX 5060, RTX 4060, RTX 3070, RTX 3060 Ti, RTX 2070, M3 · 8 GB, M2 · 8 GB, M1 · 8 GB. Mistral Nemo 12B needs about 10.7 GB at its smallest build.

Speeds are single-user decode estimates, not measurements. "ModelFit est." comes from ModelFit's model fitted on published runs (CC BY 4.0); "our est." is memory bandwidth divided by bytes read per token, times 0.55. Memory is what the best build needs, against what the machine can give a model. Method

About the model

Parameters
12B
Active per token
not in our sources
Type
not in our sources
Good for
Chat, Translation

Builds and how to run them

QuantizationLoadMin. RAMCommand
Q4_K_M9.5 GB16 GBollama run mistral-nemo:12b
Q8_012.1 GB24 GBollama run mistral-nemo:12b-instruct-2407-q8_0

Load, minimum RAM and Ollama commands: ModelFit, CC BY 4.0. Load is memory with the model loaded, before long contexts.

Questions

Can I run Mistral Nemo 12B on a Mac?

Yes on a Mac with enough memory: the smallest Apple configuration listed here that runs it at 4-bit or better is the M3 Pro · 18 GB.

How much memory does Mistral Nemo 12B need?

About 10.7 GB at the smallest build in our data, including a margin for the runtime and an 8K-token context. Longer contexts need more.

Does Mistral Nemo 12B run on a DGX Spark?

Yes, at Q4_K_M. The Spark has 128 GB of unified memory, about 121.6 GB usable.