The open-source LLM landscape is moving faster than ever. As of September 2026, there are incredible options for running powerful AI models locally — no API keys required, no per-token costs, and complete privacy. Here’s our guide to the best choices right now.
Best Overall: Qwen Family (Alibaba)
The Qwen series has become the default recommendation for local deployment, and for good reason:
- Qwen3.8-27B: Dense 27B model with excellent coding and reasoning abilities
- Qwen3.6-35B-A3B: MoE variant that activates only 3B parameters per token — blazing fast
- All under Apache 2.0 license (commercial use friendly)
- Available in sizes from 1.7B to 235B parameters
Best for Coding: Devstral & Qwen Coder Variants
If you’re building AI coding agents or need software engineering capabilities:
- Devstral (Mistral + All Hands AI): 24B parameters, Apache 2.0, built specifically for agentic software engineering
- Qwen3-Coder variants: Excellent for code generation and understanding
Best Low-Resource Option: Phi-4-mini (Microsoft)
Running on modest hardware? Microsoft’s Phi-4-mini is remarkable:
- Only 3.8B parameters
- 128K context window
- MIT license
- Runs comfortably on laptops with 8GB RAM
Best Single-GPU Option: Gemma 4 (Google)
Gemma 4 is now Apache 2.0 licensed and offers:
- Five sizes from E2B to 31B parameters
- Strong multimodal capabilities
- Up to 256K context window on larger variants
- Excellent for single-GPU workstations
For Frontier Performance: DeepSeek-V4 & Kimi K3
If you have serious hardware (multi-GPU setups):
- Kimi K3: 2.8T total parameters, 104B active — leads on agentic coding benchmarks
- DeepSeek-V4-Pro: 1.6T parameters, MIT license, excellent for long-context tasks
What About Qwen 4?
Alibaba just announced Qwen 4 at their Apsara Conference on September 22nd — four tiers including a 27B open-weights variant. Watch our dedicated article for more details as they emerge.
