The weights dropped on August 14th — Qwen3.8-27B, Apache 2.0, roughly 28 billion dense parameters, and a native 262K-token context window that surprised nobody who'd been watching the generation and everybody who hadn't.1 It even shipped with a vision encoder nobody asked for.2 The predecessor, 3.6-27B, spent the year as the model people reached for when they wanted local coding that punched above its size.3 The successor arrives carrying the 3.8 generation's training gains down to a scale that fits on one machine.
So you pull it down, and within an hour you're staring at the question that swallows every local-model thread: which quant?
That's the wrong knob. Not a wrong answer to the question — the wrong question.
The quant is the variable everyone argues about and the one that matters least for what you're actually going to do with this model. Here's the case for ignoring it, and for what to tune instead.
The quant is the variable everyone argues about, and the one that matters least.
What the measurement says
Q8_0 on an M4 Max, warm and loaded, short prompt: about 15.8 tokens per second generating, about 91 ingesting. That's the high-fidelity quant doing what it does — trading speed for the cleanest token distribution the hardware can hold. Readable, but deliberate.
You feel each response arrive.
The obvious read is Q8 for quality, live with the pace. And for one workload that read is correct. It's just correct for a workload most people running this model don't actually have — and, as the hardware section below gets into, on a machine most of them don't have either.
Two sweet spots wearing the same word
"Agents versus coding" is the framing I started with, and it's close but not quite the seam. Coding isn't on one side of the line. It's on both. The real division is between single-call fidelity and loop economics, and the same coding task can sit in either camp depending on how you run it.
A one-shot request — write me this function, refactor this class — is a single call, and it rewards quant quality directly. You want the best distribution you can afford, because there's no second attempt to average out the noise. Push toward Q6 or Q8 and take the slower tokens; you're paying for fidelity once and getting it.
An agent loop is a different animal. Plan, call a tool, read the result, decide, call the next tool, read again — a single task is ten or twenty sequential inferences stacked end to end. And on that side, the variables that dominate the experience aren't the ones quant touches. Speed compounds, because you pay the latency every turn instead of once. Tool-call formatting has to stay reliable across all twenty calls, or the agent derails. And the context has to not silently vanish.
Same model. Same domain. Opposite optimum. That's the whole thing.
The loop tax
At 16 tokens per second, a ten-turn agent task is a lot of spinner. Halving that latency is a bigger felt improvement than any marginal bump in output quality, because the slowness isn't charged once — it's charged on every hop through the loop. A quality edge you'd notice on a hard one-shot mostly disappears into the averaging of a twenty-call task. The latency doesn't average out. It accumulates.
This is why the naive advice inverts under agents. The thing you'd trade speed for on the single-call side is exactly the thing that stops paying rent once you're in a loop.
The latency doesn't average out. It accumulates.
The knob that isn't the quant
Here's where the argument gets specific, because there's a number that changes it.
Ollama defaults to a 4096-token context window. Agents feed enormous prompts — file contents, tool outputs, accumulated history — and at 4096 those prompts get silently truncated. The agent then behaves erratically for reasons that look exactly like the model is dumb and are actually the model can't see the thing you gave it.
People spend hours diagnosing a capability problem that's a configuration problem.
And the tell that it's configuration: this model has a 262K native context.1 The truncation isn't a limit of Qwen3.8-27B. It's a limit of the default, plus whatever your KV cache can hold. On a 64GB M4 Max, macOS hands the GPU about 48GB by default rather than the full 64 — so after 17GB of 4-bit weights you have roughly 31GB left for cache, not the 45GB the spec sheet implies.6 Cut it by a third and it's still enormous: the entire 262K window costs about 17GB of KV cache on this architecture. That's not "enough context."
That's the whole window, sitting switched off behind a default nobody changed.
So the context section of every local-agent setup gets written backwards. It's framed as a ceiling you keep hitting. It's actually free capability you haven't turned on. The move isn't "set num_ctx to 40K." The move is "raise it until the KV cache starts eating your headroom, then stop." The model will follow you most of the way to a quarter-million tokens if you let it and you have the memory to spend.
Notice what just happened. The two knobs that actually decide whether this model is good at agentic work — context window and keep-alive, so it doesn't unload and reload between turns — are the two nobody argues about. The argument is always the quant.
It isn't a ceiling you keep hitting. It's free capability you never switched on.
The other machine
Everything above is a Mac argument, and it doesn't survive the trip to a graphics card intact.
On unified memory, weights and KV cache draw from one pool, so "17GB of weights leaves 31GB for context" is a sentence that parses. On a discrete NVIDIA card there is no pool. There's a wall, it's printed on the box, and your 64GB of system RAM is on the other side of a PCIe bus where it does you almost no good. Spill past the wall and you don't get slower gracefully — you fall off a cliff, because every token now drags weights across that bus.
Which turns the recommendation into a question of who's actually holding what.
The July 2026 Steam survey is unkind here: 16GB just became the single most common configuration at 25.9%, 8GB sits at 25.3%, 12GB at 12.9% — and 24GB, the first tier where 17GB of weights leaves room to breathe, is 5.4%.7 Narrow it to Windows and 8GB is still the most common config in the field at 33.2%.
So the advice to run a 17GB quant is, on the NVIDIA side, advice for about one in twenty people.
On a Mac you have a memory budget. On a graphics card you have a wall.
Here's where the tiers actually land, at q8_0 KV cache:
| VRAM | Steam share | Best quant that leaves a real cache | Usable context |
|---|---|---|---|
| 8 GB | 25.3% | nothing — the smallest GGUF is 9.0 GB | — |
| 12 GB | 12.9% | UD-Q2_K_XL (10.7 GB) | ~44K |
| 16 GB | 25.9% | UD-Q3_K_XL (13.4 GB) | ~88K |
| 24 GB | 5.4% | UD-Q4_K_XL (17.9 GB) | ~208K |
| 32 GB | <1% | Q6_K (22.9 GB) | the full 262K |
One unit note, because it moves a whole tier. Hugging Face reports file sizes in decimal GB while every allocator reports memory in binary GiB, so a 13.4 GB GGUF occupies 12.48 GiB. Skip that conversion and the 16 GB row loses about 40K of context and a quantization step. It also means IQ4_XS at 15.7 GB does fit a 16 GB card — 14.62 GiB — with roughly 22K of context. Four-bit is available at the modal tier; it just costs you three quarters of your context to get there, which is a trade almost nobody should take.
The 8GB row is not a typo and not a quantization you haven't heard of. The smallest usable GGUF of this model is about 9GB. A third of Windows gamers cannot run Qwen3.8-27B on their GPU at all, and the honest advice for that tier is to run a smaller model rather than a lobotomized big one.
The Mac ladder runs the other direction — less usable memory per nominal gigabyte, because macOS only wires about 75% of it to the GPU, but it keeps climbing long after the consumer NVIDIA stack has stopped at 32GB:
| Unified memory | ~GPU budget | Best quant | Usable context |
|---|---|---|---|
| 16 GB (Air) | ~12 GiB | UD-Q2_K_XL (10.7 GB) | ~44K, and the OS suffers |
| 24 GB (Pro) | ~18 GiB | UD-Q3_K_XL (13.4 GB) | ~152K |
| 32 GB | ~24 GiB | Q4_K_M (17.1 GB) | ~232K |
| 48 GB | ~36 GiB | Q8_0 (29 GB) | the full 262K |
| 64 GB (Max) | ~48 GiB | Q8_0 (29 GB) | the full 262K |
Same model, two completely different shapes of constraint. A 24GB Mac and a 24GB graphics card are not the same machine: the card gives you more of its number but stops there, while the Mac gives you less of its number and then offers you 128GB if you pay for it.
The knob the Mac never needed
There's one control missing from every version of this argument written on Apple hardware, and it's the highest-leverage setting on a graphics card: quantize the KV cache itself.
Context isn't stored at full precision because physics demands it. --cache-type-k q8_0 --cache-type-v q8_0 halves the per-token cost of cache at a quality difference that doesn't show up in practice.8 On a 64GB Mac you never reach for it, because you were never short. On a 16GB card it's the entire ballgame — it's the difference between 44K of context and 88K, bought for nothing.
The architecture is helping here too, more than the 262K headline suggests.
This model runs hybrid attention: only 16 of its 64 layers carry a conventional KV cache, while the other 48 keep a fixed-size recurrent state that doesn't grow with context at all. That's roughly a quarter the cache cost of a normal dense 27B. The 262K window isn't generosity in the config file. It's a structural property of the model, and it's why these context numbers are achievable on hardware that has no business achieving them.
Which is worth saying plainly, because the first part of this series called its shot. That piece worked the same VRAM equation and flagged the context tax as the number most likely to be wrong first — wrong, specifically, in the direction that favors local.9 This is what that looks like arriving. It estimated 32–64K of comfortable context on a 32GB card, and on the strength of that estimate it routed agent work to cloud endpoints where prompt caching makes the cache someone else's problem. A 16GB card now holds 88K. The prediction was correct and the number it was defending was not, which is the good outcome for a structural argument and an uncomfortable one for the practical advice sitting on top of it.
So take the agent recommendation in part one as superseded for models built this way. Not because the reasoning was wrong — it was right about conventional attention, where every layer carries a cache and 128K genuinely does cost 20–40GB — but because the architecture moved underneath it. That is the thing to watch. Cache cost is now a property of model design rather than a fixed tax on context, and a design change can hand a consumer card a capability that a generation of hardware advice had already written off.
A worked example on the mainstream tier, since that's the one 26% of people are on: a 16GB card running UD-Q3_K_XL with q8_0 on both caches sits at 14.5 GiB resident with 48K of context and generates at 39.6 tokens/sec. Against the 15.8 tok/s the M4 Max turns in at Q8, that's two and a half times the speed — on a machine with a quarter of the memory. And 48K was the setting, not the ceiling: the same configuration has room for about 88K before it hits the wall.
The Mac buys context with capacity. The graphics card buys it with compression.
Loop economics said speed compounds across turns while quality averages out — which is an argument that quietly favors the graphics card, because the card's weakness is memory and its strength is exactly the thing a twenty-call agent loop charges you for twenty times. Single-call fidelity favors the Mac for the mirror-image reason: it can hold a bigger quant, and a one-shot request doesn't care that each token took longer to arrive.
The seam the piece opened between single-call and loop work turns out to run through the hardware too. Pick the side you're on before you pick anything else.
So, fine, the quant
Since you'll want the answer anyway, and it depends on the wall you're standing behind:
- Mac, 32GB and up: dynamic Q4_K_M, or Q8_0 outright at 48GB.
- Graphics card, 24GB: dynamic Q4_K_M, and this is where the standard advice finally fits.
- Graphics card, 16GB: UD-Q3_K_XL with a
q8_0cache, or IQ4_XS if you'd rather have the fourth bit than the context. - 12GB: UD-Q2_K_XL technically works at 44K, but two-bit on a dense model costs more than the context is worth — run a smaller model at higher precision. - 8GB: nothing here fits.
What makes those answers boring rather than agonizing is Unsloth's dynamic quantization, which spends its bits selectively and lands 4-bit fidelity near where a naive Q5 or Q6 sits — tool-call reliability and code coherence hold, which is precisely the property agents live or die on. The same trick is what makes the 3-bit tier viable on a 16GB card instead of merely technically loadable, and it's why the modal recommendation isn't the disaster the number suggests.4 Q8's quality edge over a good dynamic Q4 is real and, for coding specifically, marginal — not worth halving your loop speed to keep.
But read all of that for what it is. It's an answer to the question you shouldn't have led with.
The quant selection is a ten-minute decision with a low ceiling on how wrong you can go — your VRAM picks it for you far more than your judgment does. The context and keep-alive configuration is where the actual quality of the setup gets made or lost, and it's the part the threads skip.
The number I don't trust yet
Everything above rests on measured throughput and on the model behaving under a real agent loop. It does not rest on the launch table, and neither should your read of this model.
Every headline benchmark for 3.8-27B is Qwen's own. Several are in-house harnesses. The one head-to-head against a frontier proprietary model imports that model's published score rather than rerunning it under the same setup — which is a fair thing to disclose and an unfair thing to read as a controlled comparison.
None of that makes the model bad.
It makes the vendor numbers vendor numbers. The credible claim here isn't "it beats Opus on CoWorkBench."5 It's narrower and sturdier: the 3.6 predecessor earned a real reputation on real local use, the generation's training moved, and the way to know what the 27B is worth to you is your own tokens per second and your own loop watching it work.
Trust the measurement you took. Discount the table you were handed.
The split that actually matters
One honest thing the quant discussion papers over: a perfectly tuned local 27B is still meaningfully weaker at hard agentic coding than the frontier models you can rent by the token. On the genuinely difficult work, GLM-5.2 or K3 or DeepSeek V4 Pro will finish tasks the local model spins on — and Qwen's own table quietly agrees, seating the 27B mid-pack on the hard agentic-coding rows while it tops the in-house and office ones.5
Which means the real sweet spot was never a quant setting. It's a routing line.
Local Qwen for the work that's private, offline, quick, or cheap — the high-volume, low-stakes majority of an agent's day. Frontier models for the hard problems where the quality gap is the difference between done and not done. You're not picking a model. You're drawing a boundary between two of them, and the quant question lives entirely on one side of that boundary.
And that boundary is the thing worth watching, because it moves. Every local model that compresses another notch of frontier capability into 17GB pushes the line — widens the band of work where the compressed local model is simply good enough, and shrinks the band where the remote model's marginal quality justifies its latency, its cost, and handing your code to someone else's server. Qwen3.8-27B moved that line this week. It'll move again.
That's the number to track. Not the next quant.