Qwen 3.6 27B: a Local Coding Model You Can Actually Run
For most of 2025, "open-source coding model" meant choosing between two unsatisfying tiers. The small models (8B–14B) ran on your laptop and felt like working with a tired intern. The big ones — DeepSeek V3, GLM-5.1, Kimi-K2 — competed with Claude, and required a small GPU cluster to serve. (That tier has since come down to desk-side hardware: we ran a 304B model on two GB10 boxes at 88 tok/s.)
Update, October 2026. Qwen3.8-27B has since taken this model's place in the same hardware slot; our measurements of it on an RTX 5090, including the one flag that makes it 2.3 times faster, are here. The hardware guidance below still applies to the 3.6 weights.
Qwen 3.6 27B, released by Alibaba on April 22 2026, is the first open model that lands on the practical side of that gap. It runs on a single RTX 4090 or a 24 GB Mac. It gets within 4 points of Claude Opus 4.6 on SWE-bench Verified. The weights are Apache 2.0.
If you've been waiting for the moment when "self-hosted Claude Code" stops being a meme, this is it — with caveats.
What's actually new
Three things are worth knowing before you download 18 GB of weights.
It's a dense model. All 27 billion parameters fire on every token. That's the opposite of the MoE trend (Kimi, GLM, the new GPT-OSS variants), and it matters for hardware: a dense 27B fits the way you'd expect a 27B to fit. No 700B-of-which-30B-active tricks.
262K native context, extensible to 1M with YaRN. Most coding agents spend the first two minutes of a session paging in repository structure; this one can hold a mid-sized monorepo without truncation.
Thinking Preservation — reasoning that survives across turns. Toggle preserve_thinking: true and the model carries forward its prior chain-of-thought instead of regenerating it from the same context every turn. For multi-turn agentic workflows — the only kind that matter for real coding — this is the feature that bends the cost curve.
The benchmarks, with the asterisk
| Benchmark | Qwen 3.6 27B | For comparison |
|---|---|---|
| SWE-bench Verified | 77.2% | Claude Opus 4.6: 80.8% |
| Terminal-Bench 2.0 | 59.3% | Matches Claude 4.5 Opus |
| SWE-bench Pro | 53.5% | GLM-5.1 (754B MoE): 58.4% |
| SkillsBench | 48.2% | Qwen 3.5 397B: 30.0% |
The asterisk: all of these were run on Qwen's internal agent scaffold, not a neutral one. Independent reproductions are still trickling in. Treat the numbers as directional. If your evaluation depends on a specific scaffold — OpenCode, Cline, Aider's bench harness — run it yourself before claiming parity in your README.
The number that's hard to game is the one against the previous generation: 48.2% vs 30.0% on SkillsBench at one-fifteenth the parameters. Whatever Qwen learned between 3.5 and 3.6, it applied it densely.
Hardware: what you actually need
Quantized GGUF (Q4_K_M or UD-Q4_K_XL) lands at ~18 GB. That puts the practical bar at:
- Single GPU — RTX 4090, RTX 4080 Super, or any 24 GB workstation card.
- Mac — M2 Pro / M3 Pro with 24 GB unified memory or better.
- CPU + offloading — works, slowly. 64 GB system RAM, sustained around 6 tokens/sec on a recent Ryzen.
Full BF16 needs 60 GB+, which means dual-3090 or single-A6000 territory. Almost no one needs that. Q4_K_M loses roughly 1–2 points on coding benchmarks vs full precision, well within run-to-run noise.
Three ways to actually run it
1. llama.cpp — fastest path for most developers
brew install llama.cpp # or build from source
llama-server \
-hf unsloth/Qwen3.6-27B-GGUF:UD-Q4_K_XL \
--temp 0.6 --top-p 0.95 --top-k 20 \
--chat-template-kwargs '{"preserve_thinking": true}'You get an OpenAI-compatible endpoint at localhost:8080. Point any existing tool that speaks the OpenAI Chat Completions API at it and you're done. This is the path I'd recommend for 90% of readers.
2. Unsloth Studio — easiest for first-timers
A browser UI at localhost:8888 that handles weight downloads, GGUF selection, and chat-template wiring. Slower than raw llama.cpp at the margins; much faster to get running if you've never touched a local inference stack.
3. SGLang or vLLM — for serving multiple users
Version 0.5.10+ of SGLang, and recent vLLM, both ship with full Qwen 3.6 support including tool-calling and reasoning-block parsing. This is the right answer if you're serving a team rather than just yourself — batched inference on a single 24 GB card will saturate well before a single-user llama.cpp setup does.
Gotchas
A handful of small footguns are worth knowing about up front.
Avoid CUDA 13.2. It produces gibberish output on Qwen 3.6 GGUFs. 13.1 and 13.3 are fine. If you've blindly upgraded recently, downgrade before you start debugging anything else.
Ollama doesn't work yet. Qwen 3.6's vision capability ships as a separate mmproj file, and Ollama's current packaging doesn't wire it in. Watch the Ollama issue tracker; expect a fix within a release or two. Until then, llama.cpp directly.
Tool-call format. If your agent harness expects the Anthropic tool-use envelope, it won't work out of the box — Qwen ships an OpenAI-style function_call schema. Most modern harnesses (OpenCode, Aider, Cline) handle both; roll-your-own ones may need adapter code.
Should you switch from Claude or GPT?
For most production coding agents, no. Claude Opus 4.7 still leads SWE-bench at 84.3%, and the API price isn't catastrophic for any team that hasn't already optimized tokens out of its workflow.
For three specific cases, yes.
- Code that legally cannot leave your machines. Defense, healthcare, pre-IPO startups with competitive code. Self-hosting is the entire point.
- High-volume bulk operations. Migrations, codebase translations, automated refactors across a thousand repos. The token bill on the API for that kind of job is a serious chunk of an engineer's salary; a single 4090 amortizes in weeks.
- Local-first iteration. A coding agent that doesn't rate-limit you, doesn't change between sessions, and works on the plane.
Outside those cases, treat Qwen 3.6 27B as a fallback worth having configured: somewhere between 90% and 95% of Claude's output quality on most tasks, with a per-token cost of approximately zero, and the same model available six months from now without an API deprecation notice.
That's a meaningful new option. It's the first time it's been one for people running on a single GPU.
If you've benchmarked Qwen 3.6 27B on your own workflow, ai.rs would like to hear how it went. Drop a note via the contact page.