A stylized open book transforming into glowing digital code blocks, representing open-source large language models.

Best Open-Source LLMs for Local Deployment (September 2026)

The open-source model landscape in October 2026 is not a list of the biggest models. It is a list of models that fit on hardware you can actually buy, and the fit is decided by memory, not by benchmark score. This is the guide by tier, with the memory requirement for each, and where the practical limits are.

How to read this list

Two numbers decide what you can run: total parameters, which set the memory floor, and active parameters, which set the compute cost per token. A dense model uses all its parameters on every token. A mixture-of-experts model activates a small subset. A 35B MoE with 3B active needs memory for 35B but compute for 3B โ€” which is why it runs far faster than a dense 35B on the same card, and why it still needs the memory for all 35B.

Everything below assumes 4-bit quantisation, which is what makes these sizes reachable on consumer hardware. The mechanism is covered in the LLM guide.

Best overall: the Qwen family

  • Qwen3.8-27B: dense 27B, Apache 2.0, 262K context, vision encoder. About 14 GB quantised, so a 24 GB card or 32 GB of unified memory runs it with room for context.
  • Qwen3.6-35B-A3B: MoE, 3B active per token. Fast on the same hardware because the compute cost is a tenth of a dense 35B.
  • Sizes from 1.7B to 235B, all Apache 2.0, so commercial use is unambiguous.

The reason this family is the default recommendation is not raw quality. It is that the licensing is clean, the quantisation tooling is mature, and community conversions land within days of weights being published.

Best for coding: Devstral and the Qwen Coder variants

  • Devstral (Mistral with All Hands AI): 24B, Apache 2.0, built for agentic software engineering rather than chat.
  • Qwen3-Coder variants: stronger on code generation and repository-level understanding.

For an agent that edits files and runs commands, the model has to hold a repository in context across many turns. That is a memory problem before it is a quality problem: a 24B model at 4-bit plus a long context exceeds a 12 GB card. Plan for the context, not just the weights.

Best on modest hardware: Phi-4-mini

  • 3.8B parameters, MIT licence, 128K context.
  • Runs on a laptop with 8 GB of memory.

Phi-4-mini is the option that stays useful on hardware you already own. It is not competitive on frontier reasoning, but for extraction, classification, summarisation, and routing inside an automated pipeline, it is enough, and it runs at a speed larger models cannot match on the same machine.

Best single-GPU option: Gemma 4

  • Five sizes, E2B through 31B, now Apache 2.0.
  • Multimodal across the range, up to 256K context on the larger variants.

The 31B variant is the practical ceiling for a single 24 GB card. It is the tier where a workstation build stops being a hobby build.

Frontier tier: DeepSeek-V4 and Kimi K3

  • Kimi K3: 2.8T total, 104B active. Leads agentic coding benchmarks.
  • DeepSeek-V4-Pro: 1.6T parameters, MIT licence, strong on long-context work.

These are not local models in any practical sense. At 4-bit, 1.6T parameters is roughly 800 GB. They run on multi-GPU or server hardware, and the honest framing is that they are API models you can download, not models you can run.

What about Qwen 4?

Alibaba showed four Qwen 4 tiers on stage at the Apsara Conference on September 22, 2026 โ€” Max, Plus, Flash, and a 27B open-weights tier. Nothing was released. No parameter counts, no licence, no context window, no price, no benchmark. The details are in the Qwen 4 article. The 27B tier is the one that matters for local deployment, and it is the only tier that has historically shipped as open weights.

Hardware that runs these well

Models above 27B need real VRAM. The MSI GeForce RTX 5090 Gaming Trio OC (32 GB GDDR7) holds a 27B model at 4-bit with long context without offloading. The Mac mini M4 is the entry point for MLX-quantised variants. On a PC build, memory bandwidth is the second bottleneck: a G.SKILL Trident Z5 Neo 64GB DDR5 kit keeps offloaded layers usable when a model exceeds VRAM.

Which tier covers which model size is worked out in the hardware guide, and the current listing covers the same family across tiers. An engine that spreads a model across GPU, RAM, and SSD changes the answer entirely โ€” that is what Strata does, and the measured numbers are there.

2 Comments

Comments are closed