Qwen3.8-Flash-Next

Qwen3.8-Flash-Next is an experimental multimodal MoE model with 125 billion parameters (6 billion activated per token) and 51 billion N-gram embedding parameters. It previews the Qwen4 architecture and offers a systematic rethinking of attention, residual connections, and optimization. The model supports contexts up to 262,144 tokens natively and up to 1,000,000 with YaRN.

At its core is hybrid attention with a 3:1 alternation: three Gated DeltaNet (GDN) recurrent layers for every one Qwen Sparse Attention (QSA) layer. GDN compresses history into a fixed-size matrix state, updating it via a delta rule — this prevents duplicate accumulation and makes the memory robust to noise on extremely long contexts. QSA serves as the global attention in the hybrid, but unlike the full attention used in architectures starting with qwen3-next, it is not dense but sparse, and consists of a lightweight indexer (compresses keys into blocks of 4 tokens and selects the top K most relevant) and a sparse kernel that computes full attention only over the selected subset. This yields linear-to-sublinear complexity on long sequences, unlike the quadratic complexity of predecessors.

Among other innovations is an N-gram table (200 million entries, 51 billion parameters) on the host, which is asynchronously loaded and adds vectors to token representations at the second layer, improving quality with almost no computational cost and without increasing VRAM. Gated Residual (GR) expands the residual stream to four parallel branches, routing information between layers, while the Muon optimizer replaces AdamW, accelerating convergence for large matrices and ensuring training stability.

On key benchmarks, the model ranks first in its class. In agentic programming on DeepSWE 1.1, the model scores 58.7%, ahead of DeepSeek-V4-Flash-0731 (54.4%); on SWE-bench Pro — 62.5% versus 56.0% for DeepSeek and 53.4% for Claude-Opus-4.6. On the scientific benchmark GPQA Diamond, the model achieves 91.7%, surpassing all listed counterparts. In competitive programming on LiveCodeBench v6 — 91.9% versus 90.6% for DeepSeek and 88.8% for Claude. In multi-domain agentic tasks on Agents' Last Exam, the model scores 51.2 points, significantly outperforming Qwen3.7-Plus (33.6). Overall, out of 12 base tests, the model leads in 10 of them.

Thanks to its efficiency and high performance, the model is ideal for: autonomous AI agents performing complex multi-step scenarios in programming tasks and office work; and for document processing (legal contracts, scientific papers, codebases, etc.). Its multimodal capabilities should also not be overlooked, as they allow the model to be integrated into workflows requiring image and video analysis.

Announce Date: 24.08.2026
Parameters: 180B
Experts: 512
Activated at inference: 6B
Context: 263K
Layers: 48, using full attention: 12
Attention Type: Hybrid Attention
Mamba Type: Gated Delta Net
Developer: Qwen
Transformers Version: 5.8.0.dev0
vLLM Version: >=0.29.0
License: Qwen Community License 1.0

Use our pre-built public endpoints for free to test inference and explore Qwen3.8-Flash-Next capabilities. You can obtain an API access token on the token management page after registration and verification.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.