FitLocally by GPUwerk

NVIDIA · 120B · MoE

Can you run Nemotron 3.5 Super VL 120B-A12B locally?

Nemotron 3.5 Super VL 120B-A12B is a mixture-of-experts model with 120B parameters, 12B active per token. At 4-bit or better it runs on 19 of the 120 hardware configurations listed here; the smallest is the RTX PRO 6000 Blackwell (96 GB). On a 128 GB DGX Spark it runs at Q6_K, at roughly 15 tokens per second (our est.).

Data as of 2 November 2026

Fastest machines for Nemotron 3.5 Super VL 120B-A12B

Every machine that runs it, fastest first. Filter by type; your machine is marked. Compare machines running it side by side

  1. 12x RTX 5090Graphics card · 64 GB~168 tok/s our est.Q3_K_M · 61 of 62 GB 3-bit only
  2. 2RTX PRO 6000 BlackwellGraphics card · 96 GB~135 tok/s our est.Q4_K_M · 76 of 95 GB
  3. 3M2 UltraMac · 64 GB · 2 variants~98 tok/s our est.Q2_K · 47 GB 2-bit only
  4. 4M1 UltraMac · 64 GB · 2 variants~98 tok/s our est.Q2_K · 47 GB 2-bit only
  5. 52x RTX PRO 6000 BlackwellGraphics card · 192 GB~77 tok/s our est.Q8_0 · 132 of 190 GB
  6. 6M3 UltraMac · 96 GB · 2 variants~77 tok/s our est.Q3_K_M · 61 GB 3-bit only
  7. 7M2 Ultra · 96 GBMac · 96 GB~75 tok/s our est.Q3_K_M · 61 of 72 GB 3-bit only
  8. 8M4 Max · 64 GB · 40cMac · 64 GB~67 tok/s our est.Q2_K · 47 of 48 GB 2-bit only
  9. 9M2 UltraMac · 128 GB · 2 variants~60 tok/s our est.Q4_K_M · 76 GB
  10. 10M1 UltraMac · 128 GB · 2 variants~60 tok/s our est.Q4_K_M · 76 GB
  11. 11M3 Max · 64 GB · 40cMac · 64 GB~49 tok/s our est.Q2_K · 47 of 48 GB 2-bit only
  12. 12M2 MaxMac · 64 GB · 2 variants~49 tok/s our est.Q2_K · 47 GB 2-bit only
  13. 13M1 MaxMac · 64 GB · 2 variants~49 tok/s our est.Q2_K · 47 GB 2-bit only
  14. 14M4 Max · 128 GB · 40cMac · 128 GB~41 tok/s our est.Q4_K_M · 76 of 96 GB
  15. 15M2 Max · 96 GB · 38cMac · 96 GB~38 tok/s our est.Q3_K_M · 61 of 72 GB 3-bit only
  16. 16M3 UltraMac · 256 GB · 2 variants~35 tok/s our est.Q8_0 · 132 GB
  17. 17M2 UltraMac · 192 GB · 2 variants~35 tok/s our est.Q8_0 · 132 GB
  18. 18M4 ProMac · 64 GB · 2 variants~33 tok/s our est.Q2_K · 47 GB 2-bit only
  19. 19Ryzen AI Max+ 395 · 64 GBAI mini PC · 64 GB~31 tok/s our est.Q2_K · 47 of 48 GB 2-bit only
  20. 20M3 Max · 128 GB · 40cMac · 128 GB~30 tok/s our est.Q4_K_M · 76 of 96 GB
  21. 21M3 Max · 96 GB · 30cMac · 96 GB~28 tok/s our est.Q3_K_M · 61 of 72 GB 3-bit only
  22. 22Ryzen AI Max+ 395 · 128 GBAI mini PC · 128 GB~19 tok/s our est.Q4_K_M · 76 of 96 GB
  23. 23M3 Ultra · 512 GB · 80cMac · 512 GB~19 tok/s our est.FP16 · 247 of 384 GB
  24. 24DGX Spark (GB10)AI mini PC · 128 GB~15 tok/s our est.Q6_K · 102 of 122 GB
  25. 25M5 Max · 64 GB · 40cMac · 64 GBno estimateQ2_K · 47 of 48 GB 2-bit only
  26. 26M5 ProMac · 64 GB · 2 variantsno estimateQ2_K · 47 GB 2-bit only
  27. 27M5 UltraMac · 96 GB · 2 variantsno estimateQ3_K_M · 61 GB 3-bit only
  28. 28M5 Max · 128 GB · 40cMac · 128 GBno estimateQ4_K_M · 76 of 96 GB
  29. 29M5 UltraMac · 256–512 GB · 3 variantsno estimateQ4_K_M · 76 GB
Too big for 77 machines

2x RTX 4090, 2x RTX 3090 Ti, 2x RTX 3090, 2x RX 7900 XTX, M5 Max · 48 GB · 40c, M4 Max · 48 GB · 40c, M3 Max · 48 GB · 40c, M5 Pro · 48 GB · 20c, M5 Pro · 48 GB · 16c, M4 Pro · 48 GB · 20c, M4 Pro · 48 GB · 16c, RTX 5090, M5 Max · 36 GB · 32c, M4 Max · 36 GB · 32c, M3 Max · 36 GB · 30c, M3 Pro · 36 GB, RTX 4090, RTX 3090 Ti, RTX 3090, RX 7900 XTX, M2 Max · 32 GB · 38c, M2 Max · 32 GB · 30c, M1 Max · 32 GB · 32c, M1 Max · 32 GB · 24c, M2 Pro · 32 GB, M1 Pro · 32 GB, M6 · 32 GB · 12c, M5 · 32 GB · 10c, M5 · 32 GB · 8c, M4 · 32 GB, RX 7900 XT, M5 Pro · 24 GB · 20c, M5 Pro · 24 GB · 16c, M4 Pro · 24 GB · 20c, M4 Pro · 24 GB · 16c, M6 · 24 GB · 12c, M6 · 24 GB · 10c, M5 · 24 GB · 10c, M5 · 24 GB · 8c, M4 · 24 GB, M3 · 24 GB, M2 · 24 GB, RTX 5080, RTX 5070 Ti, RTX 5060 Ti, RTX 4080 SUPER, RTX 4080, RTX 4070 Ti SUPER, RTX 4060 Ti, Tesla T4, Tesla P100, RX 9070 XT, RX 9060 XT, M3 Pro · 18 GB, RTX 5070, RTX 4070 Ti, RTX 4070 SUPER, RTX 4070, RTX 3060, M2 Pro · 16 GB, M1 Pro · 16 GB, M6 · 16 GB · 12c, M6 · 16 GB · 10c, M5 · 16 GB · 10c, M5 · 16 GB · 8c, M4 · 16 GB, M3 · 16 GB, M2 · 16 GB, M1 · 16 GB, RTX 5060, RTX 4060, RTX 3070, RTX 3060 Ti, RTX 2070, M3 · 8 GB, M2 · 8 GB, M1 · 8 GB. Nemotron 3.5 Super VL 120B-A12B needs about 76 GB at its smallest build.

Speeds are single-user decode estimates, not measurements. "ModelFit est." comes from ModelFit's model fitted on published runs (CC BY 4.0); "our est." is memory bandwidth divided by bytes read per token, times 0.55. Memory is what the best build needs, against what the machine can give a model. Method

DGX Spark setup notes: GPUwerk guide for Nemotron 3.5 Super VL 120B-A12B.

About the model

Parameters
120B
Active per token
12B
Type
Mixture of experts
Good for
Vision-language MoE
Model card
Hugging Face
Hugging Face count
124.45B (agrees)

Questions

Can I run Nemotron 3.5 Super VL 120B-A12B on a Mac?

Yes on a Mac with enough memory: the smallest Apple configuration listed here that runs it at 4-bit or better is the M2 Ultra · 128 GB · 76c.

How much memory does Nemotron 3.5 Super VL 120B-A12B need?

About 47 GB at the smallest build in our data, including a margin for the runtime and an 8K-token context. Longer contexts need more.

Does Nemotron 3.5 Super VL 120B-A12B run on a DGX Spark?

Yes, at Q6_K. The Spark has 128 GB of unified memory, about 121.6 GB usable.