Llama 4 Scout Lands: What a 10M Context Window Actually Buys You
Meta's new mixture-of-experts flagship is open-weights, agent-tuned, and absurdly long-context. We break down the architecture and what it means for self-hosters.
Meta released Llama 4 Scout under open weights, and it is the most consequential open release of the year. A mixture-of-experts architecture, a 10M-token context window, and explicit tuning for agentic tool use.
#Architecture at a glance
Scout uses a sparse MoE design: a large total parameter count with a much smaller active footprint per token. That is the trick that makes a 10M window economically plausible at all.
- Sparse MoE with shared expert layers
- 10M-token context via interleaved attention
- Tool-use and structured-output tuning baked into post-training
- Single-GPU inference possible at quantized precision
#What the context window buys you
The honest answer: fewer moving parts. Teams that currently stitch together retrieval, chunking, and reranking pipelines can collapse much of that into a single long-context call.
#Self-hosting math
At 4-bit quantization, Scout fits on a single high-memory GPU for modest batch sizes. For production throughput, expect a small cluster — but the cost curve is dramatically better than dense equivalents.
#Bottom line
Scout is the first open model where the context window is the headline feature rather than the benchmark score. That is a sign of where the whole field is heading.
See the full entry on the model radar.
// share this dispatch
// the signal
One email. The week's sharpest AI analysis.
Every Friday: the dispatches that mattered, the models that shipped, and the one chart you need to see. No spam, no filler — unsubscribe anytime.
Join 2,400+ researchers and engineers. Unsubscribe anytime.