Nemotron · 30B · MoE
Can you run Nemotron 3.5 Lightning 30B-A3B locally?
Nemotron 3.5 Lightning 30B-A3B is a mixture-of-experts model with 30B parameters, 3B active per token. At 4-bit or better it runs on 73 of the 120 hardware configurations listed here; the smallest is the RTX 4090 (24 GB). On a 128 GB DGX Spark it runs at Q4_K_M, at roughly 59 tokens per second (our est.).
Data as of 2 November 2026
Fastest machines for Nemotron 3.5 Lightning 30B-A3B
Every machine that runs it, fastest first. Filter by type; your machine is marked. Compare machines running it side by side
- 1RTX 4090Graphics card · 24 GB~293 tok/s our est.IQ4_XS · 20 of 23 GB
- 2RTX 3090 TiGraphics card · 24 GB~293 tok/s our est.IQ4_XS · 20 of 23 GB
- 3RX 7900 XTXGraphics card · 24 GB~279 tok/s our est.IQ4_XS · 20 of 23 GB
- 4RTX 3090Graphics card · 24 GB~272 tok/s our est.IQ4_XS · 20 of 23 GB
- 5RTX 5090Graphics card · 32 GB~250 tok/s ModelFit est.Q4_K_M · 27 of 31 GB
- 6RTX PRO 6000 BlackwellGraphics card · 96 GB~250 tok/s ModelFit est.Q4_K_M · 27 of 95 GB
- 72x RTX 5090Graphics card · 64 GB~233 tok/s ModelFit est.Q4_K_M · 27 of 62 GB
- 82x RTX PRO 6000 BlackwellGraphics card · 192 GB~233 tok/s ModelFit est.Q4_K_M · 27 of 190 GB
- 9M5 UltraMac · 96–512 GB · 5 variants~160 tok/s ModelFit est.Q4_K_M · 27 GB
- 102x RTX 4090Graphics card · 48 GB~153 tok/s ModelFit est.Q4_K_M · 27 of 46 GB
- 112x RTX 3090 TiGraphics card · 48 GB~143 tok/s ModelFit est.Q4_K_M · 27 of 46 GB
- 122x RTX 3090Graphics card · 48 GB~135 tok/s ModelFit est.Q4_K_M · 27 of 46 GB
- 13M4 Max · 36 GB · 32cMac · 36 GB~119 tok/s our est.IQ4_XS · 20 of 24 GB
- 142x RX 7900 XTXGraphics card · 48 GB~117 tok/s ModelFit est.Q4_K_M · 27 of 46 GB
- 15M2 MaxMac · 32 GB · 2 variants~116 tok/s our est.IQ4_XS · 20 GB
- 16M1 MaxMac · 32 GB · 2 variants~116 tok/s our est.IQ4_XS · 20 GB
- 17M3 UltraMac · 96–512 GB · 5 variants~112 tok/s ModelFit est.Q4_K_M · 27 GB
- 18M5 MaxMac · 48–128 GB · 3 variants~103 tok/s ModelFit est.Q4_K_M · 27 GB
- 19M2 UltraMac · 64–192 GB · 7 variants~99 tok/s ModelFit est.Q4_K_M · 27 GB
- 20M1 UltraMac · 64–128 GB · 4 variants~88 tok/s ModelFit est.Q4_K_M · 27 GB
- 21M3 Max · 36 GB · 30cMac · 36 GB~87 tok/s our est.IQ4_XS · 20 of 24 GB
- 22M4 MaxMac · 48–128 GB · 3 variants~86 tok/s ModelFit est.Q4_K_M · 27 GB
- 23M3 MaxMac · 48–128 GB · 3 variants~68 tok/s ModelFit est.Q4_K_M · 27 GB
- 24M2 MaxMac · 64–96 GB · 3 variants~63 tok/s ModelFit est.Q4_K_M · 27 GB
- 25M5 ProMac · 48–64 GB · 4 variants~60 tok/s ModelFit est.Q4_K_M · 27 GB
- 26DGX Spark (GB10)AI mini PC · 128 GB~59 tok/s our est.Q4_K_M · 27 of 122 GB
- 27M2 Pro · 32 GBMac · 32 GB~58 tok/s our est.IQ4_XS · 20 of 21 GB
- 28M1 Pro · 32 GBMac · 32 GB~58 tok/s our est.IQ4_XS · 20 of 21 GB
- 29M1 MaxMac · 64 GB · 2 variants~57 tok/s ModelFit est.Q4_K_M · 27 GB
- 30Ryzen AI Max+ 395 · 64 GBAI mini PC · 64 GB~55 tok/s our est.Q4_K_M · 27 of 48 GB
- 31M3 Max · 96 GB · 30cMac · 96 GB~52 tok/s ModelFit est.Q4_K_M · 27 of 72 GB
- 32M4 ProMac · 48–64 GB · 4 variants~50 tok/s ModelFit est.Q4_K_M · 27 GB
- 33M5Mac · 32 GB · 2 variants~45 tok/s our est.IQ4_XS · 20 GB
- 34M3 Pro · 36 GBMac · 36 GB~44 tok/s our est.IQ4_XS · 20 of 24 GB
- 35Ryzen AI Max+ 395 · 128 GBAI mini PC · 128 GB~40 tok/s ModelFit est.Q4_K_M · 27 of 96 GB
- 36M4 · 32 GBMac · 32 GB~35 tok/s our est.IQ4_XS · 20 of 21 GB
- 37M6 · 32 GB · 12cMac · 32 GBno estimateIQ4_XS · 20 of 21 GB
- 38M5 Max · 36 GB · 32cMac · 36 GBno estimateIQ4_XS · 20 of 24 GB
Too big for 47 machines
RX 7900 XT, M5 Pro · 24 GB · 20c, M5 Pro · 24 GB · 16c, M4 Pro · 24 GB · 20c, M4 Pro · 24 GB · 16c, M6 · 24 GB · 12c, M6 · 24 GB · 10c, M5 · 24 GB · 10c, M5 · 24 GB · 8c, M4 · 24 GB, M3 · 24 GB, M2 · 24 GB, RTX 5080, RTX 5070 Ti, RTX 5060 Ti, RTX 4080 SUPER, RTX 4080, RTX 4070 Ti SUPER, RTX 4060 Ti, Tesla T4, Tesla P100, RX 9070 XT, RX 9060 XT, M3 Pro · 18 GB, RTX 5070, RTX 4070 Ti, RTX 4070 SUPER, RTX 4070, RTX 3060, M2 Pro · 16 GB, M1 Pro · 16 GB, M6 · 16 GB · 12c, M6 · 16 GB · 10c, M5 · 16 GB · 10c, M5 · 16 GB · 8c, M4 · 16 GB, M3 · 16 GB, M2 · 16 GB, M1 · 16 GB, RTX 5060, RTX 4060, RTX 3070, RTX 3060 Ti, RTX 2070, M3 · 8 GB, M2 · 8 GB, M1 · 8 GB. Nemotron 3.5 Lightning 30B-A3B needs about 20 GB at its smallest build.
Speeds are single-user decode estimates, not measurements. "ModelFit est." comes from ModelFit's model fitted on published runs (CC BY 4.0); "our est." is memory bandwidth divided by bytes read per token, times 0.55. Memory is what the best build needs, against what the machine can give a model. Method
About the model
- Parameters
- 30B
- Active per token
- 3B
- Type
- Mixture of experts
- Good for
- Agentic, Coding, Long context
Builds and how to run them
| Quantization | Load | Min. RAM | Command |
|---|---|---|---|
| Q4_K_M | 25 GB | 36 GB | ollama run nemotron-3.5-lightning:30b |
| IQ4_XS | 18.9 GB | 32 GB | GGUF or MLX build |
Load, minimum RAM and Ollama commands: ModelFit, CC BY 4.0. Load is memory with the model loaded, before long contexts.
Questions
Can I run Nemotron 3.5 Lightning 30B-A3B on a Mac?
Yes on a Mac with enough memory: the smallest Apple configuration listed here that runs it at 4-bit or better is the M2 Max · 32 GB · 38c.
How much memory does Nemotron 3.5 Lightning 30B-A3B need?
About 20 GB at the smallest build in our data, including a margin for the runtime and an 8K-token context. Longer contexts need more.
Does Nemotron 3.5 Lightning 30B-A3B run on a DGX Spark?
Yes, at Q4_K_M. The Spark has 128 GB of unified memory, about 121.6 GB usable.