FitLocally by GPUwerk

NVIDIA · 9B

Can you run Nemotron Nano 9B v2 locally?

Nemotron Nano 9B v2 is a dense model with 9B parameters. At 4-bit or better it runs on 117 of the 120 hardware configurations listed here; the smallest is the RTX 5060 (8 GB). On a 128 GB DGX Spark it runs at Q8_0, at roughly 16 tokens per second (our est.).

Data as of 2 November 2026

Fastest machines for Nemotron Nano 9B v2

Every machine that runs it, fastest first. Filter by type; your machine is marked. Compare machines running it side by side

  1. 1RTX 5080Graphics card · 16 GB~55 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  2. 2RTX 5090Graphics card · 32 GB~55 tok/s our est.FP16 · 19.4 of 31 GB
  3. 32x RTX 5090Graphics card · 64 GB~55 tok/s our est.FP16 · 19.4 of 62 GB
  4. 4RTX PRO 6000 BlackwellGraphics card · 96 GB~55 tok/s our est.FP16 · 19.4 of 95 GB
  5. 52x RTX PRO 6000 BlackwellGraphics card · 192 GB~55 tok/s our est.FP16 · 19.4 of 190 GB
  6. 6RTX 5070 TiGraphics card · 16 GB~52 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  7. 7RX 7900 XTGraphics card · 20 GB~46 tok/s our est.Q8_0 · 10.8 of 19.0 GB
  8. 8RTX 5060Graphics card · 8 GB~45 tok/s our est.Q4_K_M · 6.6 of 7.0 GB
  9. 9RTX 3070Graphics card · 8 GB~45 tok/s our est.Q4_K_M · 6.6 of 7.0 GB
  10. 10RTX 3060 TiGraphics card · 8 GB~45 tok/s our est.Q4_K_M · 6.6 of 7.0 GB
  11. 11RTX 2070Graphics card · 8 GB~45 tok/s our est.Q4_K_M · 6.6 of 7.0 GB
  12. 12RTX 4080 SUPERGraphics card · 16 GB~42 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  13. 13Tesla P100Graphics card · 16 GB~42 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  14. 14RTX 4080Graphics card · 16 GB~41 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  15. 15RTX 5070Graphics card · 12 GB~39 tok/s our est.Q8_0 · 10.8 of 11.0 GB
  16. 16RTX 4070 Ti SUPERGraphics card · 16 GB~39 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  17. 17RX 9070 XTGraphics card · 16 GB~37 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  18. 18RTX 4090Graphics card · 24 GB~31 tok/s our est.FP16 · 19.4 of 23 GB
  19. 19RTX 3090 TiGraphics card · 24 GB~31 tok/s our est.FP16 · 19.4 of 23 GB
  20. 202x RTX 4090Graphics card · 48 GB~31 tok/s our est.FP16 · 19.4 of 46 GB
  21. 212x RTX 3090 TiGraphics card · 48 GB~31 tok/s our est.FP16 · 19.4 of 46 GB
  22. 22RX 7900 XTXGraphics card · 24 GB~29 tok/s our est.FP16 · 19.4 of 23 GB
  23. 232x RX 7900 XTXGraphics card · 48 GB~29 tok/s our est.FP16 · 19.4 of 46 GB
  24. 24RTX 4070 TiGraphics card · 12 GB~29 tok/s our est.Q8_0 · 10.8 of 11.0 GB
  25. 25RTX 4070 SUPERGraphics card · 12 GB~29 tok/s our est.Q8_0 · 10.8 of 11.0 GB
  26. 26RTX 4070Graphics card · 12 GB~29 tok/s our est.Q8_0 · 10.8 of 11.0 GB
  27. 27RTX 3090Graphics card · 24 GB~29 tok/s our est.FP16 · 19.4 of 23 GB
  28. 282x RTX 3090Graphics card · 48 GB~29 tok/s our est.FP16 · 19.4 of 46 GB
  29. 29RTX 4060Graphics card · 8 GB~27 tok/s our est.Q4_K_M · 6.6 of 7.0 GB
  30. 30RTX 5060 TiGraphics card · 16 GB~26 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  31. 31M3 UltraMac · 96–512 GB · 5 variants~25 tok/s our est.FP16 · 19.4 GB
  32. 32M2 UltraMac · 64–192 GB · 7 variants~24 tok/s our est.FP16 · 19.4 GB
  33. 33M1 UltraMac · 64–128 GB · 4 variants~24 tok/s our est.FP16 · 19.4 GB
  34. 34RTX 3060Graphics card · 12 GB~21 tok/s our est.Q8_0 · 10.8 of 11.0 GB
  35. 35Tesla T4Graphics card · 16 GB~18 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  36. 36RX 9060 XTGraphics card · 16 GB~18 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  37. 37M3 MaxMac · 36–96 GB · 2 variants~17 tok/s our est.Q8_0 · 10.8 GB
  38. 38M4 MaxMac · 48–128 GB · 3 variants~17 tok/s our est.FP16 · 19.4 GB
  39. 39RTX 4060 TiGraphics card · 16 GB~17 tok/s our est.Q8_0 · 10.8 of 15.0 GB
  40. 40M3 · 8 GBMac · 8 GB~16 tok/s our est.Q2_K · 4.5 of 5.3 GB 2-bit only
  41. 41M2 · 8 GBMac · 8 GB~16 tok/s our est.Q2_K · 4.5 of 5.3 GB 2-bit only
  42. 42M4 ProMac · 24–64 GB · 6 variants~16 tok/s our est.Q8_0 · 10.8 GB
  43. 43DGX Spark (GB10)AI mini PC · 128 GB~16 tok/s our est.Q8_0 · 10.8 of 122 GB
  44. 44M2 Pro · 16 GBMac · 16 GB~15 tok/s our est.Q6_K · 8.6 of 10.7 GB
  45. 45M1 Pro · 16 GBMac · 16 GB~15 tok/s our est.Q6_K · 8.6 of 10.7 GB
  46. 46Ryzen AI Max+ 395AI mini PC · 64–128 GB · 2 variants~15 tok/s our est.Q8_0 · 10.8 GB
  47. 47M4 Max · 36 GB · 32cMac · 36 GB~13 tok/s our est.FP16 · 19.4 of 24 GB
  48. 48M2 MaxMac · 32–96 GB · 5 variants~12 tok/s our est.FP16 · 19.4 GB
  49. 49M1 MaxMac · 32–64 GB · 4 variants~12 tok/s our est.FP16 · 19.4 GB
  50. 50M3 MaxMac · 48–128 GB · 3 variants~12 tok/s our est.FP16 · 19.4 GB
  51. 51M4Mac · 16–32 GB · 3 variants~12 tok/s our est.Q4_K_M · 6.6 GB
  52. 52M2 Pro · 32 GBMac · 32 GB~12 tok/s our est.Q8_0 · 10.8 of 21 GB
  53. 53M1 Pro · 32 GBMac · 32 GB~12 tok/s our est.Q8_0 · 10.8 of 21 GB
  54. 54M5Mac · 16–32 GB · 6 variants~11 tok/s our est.Q6_K · 8.6 GB
  55. 55M3 ProMac · 18–36 GB · 2 variants~11 tok/s our est.Q6_K · 8.6 GB
  56. 56M1 · 8 GBMac · 8 GB~11 tok/s our est.Q2_K · 4.5 of 5.3 GB 2-bit only
  57. 57M3Mac · 16–24 GB · 2 variants~10 tok/s our est.Q4_K_M · 6.6 GB
  58. 58M2Mac · 16–24 GB · 2 variants~10 tok/s our est.Q4_K_M · 6.6 GB
  59. 59M1 · 16 GBMac · 16 GB~6.9 tok/s slow our est.Q4_K_M · 6.6 of 10.7 GB
  60. 60M6Mac · 16–32 GB · 5 variantsno estimateQ4_K_M · 6.6 GB
  61. 61M5 ProMac · 24–64 GB · 6 variantsno estimateQ4_K_M · 6.6 GB
  62. 62M5 MaxMac · 36–128 GB · 4 variantsno estimateQ4_K_M · 6.6 GB
  63. 63M5 UltraMac · 96–512 GB · 5 variantsno estimateQ4_K_M · 6.6 GB

Speeds are single-user decode estimates, not measurements. "ModelFit est." comes from ModelFit's model fitted on published runs (CC BY 4.0); "our est." is memory bandwidth divided by bytes read per token, times 0.55. Memory is what the best build needs, against what the machine can give a model. Method

DGX Spark setup notes: GPUwerk guide for Nemotron Nano 9B v2.

About the model

Parameters
9B
Active per token
9B
Type
Dense
Good for
Small fast model, draft model
Model card
Hugging Face

Questions

Can I run Nemotron Nano 9B v2 on a Mac?

Yes on a Mac with enough memory: the smallest Apple configuration listed here that runs it at 4-bit or better is the M2 Pro · 16 GB.

How much memory does Nemotron Nano 9B v2 need?

About 4.5 GB at the smallest build in our data, including a margin for the runtime and an 8K-token context. Longer contexts need more.

Does Nemotron Nano 9B v2 run on a DGX Spark?

Yes, at Q8_0. The Spark has 128 GB of unified memory, about 121.6 GB usable.