FitLocally by GPUwerk

Mistral · 24B

Can you run Mistral Small 3.1 locally?

Mistral Small 3.1 is a dense model with 24B parameters. At 4-bit or better it runs on 74 of the 120 hardware configurations listed here; the smallest is the RX 7900 XT (20 GB). On a 128 GB DGX Spark it runs at Q4_K_M, at roughly 10 tokens per second (our est.).

Data as of 2 November 2026

Fastest machines for Mistral Small 3.1

Every machine that runs it, fastest first. Filter by type; your machine is marked. Compare machines running it side by side

  1. 1RTX 5090Graphics card · 32 GB~58 tok/s ModelFit est.Q8_0 · 25 of 31 GB
  2. 2RTX PRO 6000 BlackwellGraphics card · 96 GB~58 tok/s ModelFit est.Q8_0 · 25 of 95 GB
  3. 32x RTX 5090Graphics card · 64 GB~57 tok/s ModelFit est.Q8_0 · 25 of 62 GB
  4. 42x RTX PRO 6000 BlackwellGraphics card · 192 GB~57 tok/s ModelFit est.Q8_0 · 25 of 190 GB
  5. 5RTX 4090Graphics card · 24 GB~37 tok/s our est.Q4_K_M · 16.4 of 23 GB
  6. 6RTX 3090 TiGraphics card · 24 GB~37 tok/s our est.Q4_K_M · 16.4 of 23 GB
  7. 7M5 UltraMac · 96–512 GB · 5 variants~36 tok/s ModelFit est.Q8_0 · 25 GB
  8. 8RX 7900 XTXGraphics card · 24 GB~35 tok/s our est.Q4_K_M · 16.4 of 23 GB
  9. 9RTX 3090Graphics card · 24 GB~34 tok/s our est.Q4_K_M · 16.4 of 23 GB
  10. 102x RTX 4090Graphics card · 48 GB~32 tok/s ModelFit est.Q8_0 · 25 of 46 GB
  11. 112x RTX 3090 TiGraphics card · 48 GB~31 tok/s ModelFit est.Q8_0 · 25 of 46 GB
  12. 12RX 7900 XTGraphics card · 20 GB~29 tok/s our est.Q4_K_M · 16.4 of 19.0 GB
  13. 132x RTX 3090Graphics card · 48 GB~29 tok/s ModelFit est.Q8_0 · 25 of 46 GB
  14. 142x RX 7900 XTXGraphics card · 48 GB~28 tok/s ModelFit est.Q8_0 · 25 of 46 GB
  15. 15M3 UltraMac · 96–512 GB · 5 variants~24 tok/s ModelFit est.Q8_0 · 25 GB
  16. 16M2 UltraMac · 64–192 GB · 7 variants~22 tok/s ModelFit est.Q8_0 · 25 GB
  17. 17M1 UltraMac · 64–128 GB · 4 variants~20 tok/s ModelFit est.Q8_0 · 25 GB
  18. 18M5 MaxMac · 48–128 GB · 3 variants~19 tok/s ModelFit est.Q8_0 · 25 GB
  19. 19M4 MaxMac · 48–128 GB · 3 variants~16 tok/s ModelFit est.Q8_0 · 25 GB
  20. 20M4 Max · 36 GB · 32cMac · 36 GB~15 tok/s our est.Q4_K_M · 16.4 of 24 GB
  21. 21M2 MaxMac · 32 GB · 2 variants~15 tok/s our est.Q4_K_M · 16.4 GB
  22. 22M1 MaxMac · 32 GB · 2 variants~15 tok/s our est.Q4_K_M · 16.4 GB
  23. 23M3 MaxMac · 48–128 GB · 3 variants~12 tok/s ModelFit est.Q8_0 · 25 GB
  24. 24M2 MaxMac · 64–96 GB · 3 variants~12 tok/s ModelFit est.Q8_0 · 25 GB
  25. 25M3 MaxMac · 36–96 GB · 2 variants~11 tok/s our est.Q4_K_M · 16.4 GB
  26. 26M1 MaxMac · 64 GB · 2 variants~11 tok/s ModelFit est.Q8_0 · 25 GB
  27. 27M5 ProMac · 48–64 GB · 4 variants~10 tok/s ModelFit est.Q8_0 · 25 GB
  28. 28M4 ProMac · 48–64 GB · 4 variants~10 tok/s our est.Q4_K_M · 16.4 GB
  29. 29DGX Spark (GB10)AI mini PC · 128 GB~10 tok/s our est.Q4_K_M · 16.4 of 122 GB
  30. 30Ryzen AI Max+ 395AI mini PC · 64–128 GB · 2 variants~9.4 tok/s slow our est.Q4_K_M · 16.4 GB
  31. 31M2 Pro · 32 GBMac · 32 GB~7.3 tok/s slow our est.Q4_K_M · 16.4 of 21 GB
  32. 32M1 Pro · 32 GBMac · 32 GB~7.3 tok/s slow our est.Q4_K_M · 16.4 of 21 GB
  33. 33M5Mac · 32 GB · 2 variants~5.6 tok/s slow our est.Q4_K_M · 16.4 GB
  34. 34M3 Pro · 36 GBMac · 36 GB~5.5 tok/s slow our est.Q4_K_M · 16.4 of 24 GB
  35. 35M4 · 32 GBMac · 32 GB~4.4 tok/s slow our est.Q4_K_M · 16.4 of 21 GB
  36. 36M6 · 32 GB · 12cMac · 32 GBno estimateQ4_K_M · 16.4 of 21 GB
  37. 37M5 Max · 36 GB · 32cMac · 36 GBno estimateQ4_K_M · 16.4 of 24 GB
Too big for 46 machines

M5 Pro · 24 GB · 20c, M5 Pro · 24 GB · 16c, M4 Pro · 24 GB · 20c, M4 Pro · 24 GB · 16c, M6 · 24 GB · 12c, M6 · 24 GB · 10c, M5 · 24 GB · 10c, M5 · 24 GB · 8c, M4 · 24 GB, M3 · 24 GB, M2 · 24 GB, RTX 5080, RTX 5070 Ti, RTX 5060 Ti, RTX 4080 SUPER, RTX 4080, RTX 4070 Ti SUPER, RTX 4060 Ti, Tesla T4, Tesla P100, RX 9070 XT, RX 9060 XT, M3 Pro · 18 GB, RTX 5070, RTX 4070 Ti, RTX 4070 SUPER, RTX 4070, RTX 3060, M2 Pro · 16 GB, M1 Pro · 16 GB, M6 · 16 GB · 12c, M6 · 16 GB · 10c, M5 · 16 GB · 10c, M5 · 16 GB · 8c, M4 · 16 GB, M3 · 16 GB, M2 · 16 GB, M1 · 16 GB, RTX 5060, RTX 4060, RTX 3070, RTX 3060 Ti, RTX 2070, M3 · 8 GB, M2 · 8 GB, M1 · 8 GB. Mistral Small 3.1 needs about 16.4 GB at its smallest build.

Speeds are single-user decode estimates, not measurements. "ModelFit est." comes from ModelFit's model fitted on published runs (CC BY 4.0); "our est." is memory bandwidth divided by bytes read per token, times 0.55. Memory is what the best build needs, against what the machine can give a model. Method

About the model

Parameters
24B
Active per token
24B
Type
Dense
Good for
Chat, Coding

Builds and how to run them

QuantizationLoadMin. RAMCommand
Q4_K_M15.0 GB24 GBollama run mistral-small3.1:24b
Q8_023 GB36 GBollama run mistral-small3.1:24b-instruct-2503-q8_0

Load, minimum RAM and Ollama commands: ModelFit, CC BY 4.0. Load is memory with the model loaded, before long contexts.

Questions

Can I run Mistral Small 3.1 on a Mac?

Yes on a Mac with enough memory: the smallest Apple configuration listed here that runs it at 4-bit or better is the M2 Max · 32 GB · 38c.

How much memory does Mistral Small 3.1 need?

About 16.4 GB at the smallest build in our data, including a margin for the runtime and an 8K-token context. Longer contexts need more.

Does Mistral Small 3.1 run on a DGX Spark?

Yes, at Q4_K_M. The Spark has 128 GB of unified memory, about 121.6 GB usable.