Qwen3-Next-80B-A3B-Instruct

Qwen3-Next-80B-A3B-Instruct is the first model based on the innovative Qwen3-Next architecture, which incorporates a series of technological breakthroughs developed by its creators. At its core lies a hybrid attention system that combines two mechanisms in a 3:1 ratio: Gated DeltaNet (Linear Attention used in 75% of layers) enables linear computational complexity and highly efficient processing of long sequences, while Gated Attention (Full Attention in the remaining 25% of layers) ensures high accuracy and strong information retrieval capabilities. This architecture addresses a fundamental limitation of traditional attention—linear attention is fast but weak in retrieval tasks, whereas standard attention is computationally expensive and slow during inference. Their hybrid combination demonstrates superior learning and contextual understanding abilities compared to alternative approaches such as Sliding Window Attention or Mamba2.

The model also implements an ultra-sparse Mixture-of-Experts (MoE) architecture with 512 experts, of which only 10 routed experts plus 1 shared expert are activated—just 3.7% of the total parameter count. Compared to the MoE structure in Qwen3 (128 experts, 8 active), this represents a significant advancement in efficiency and scalability. Qwen3-Next introduces several critical optimizations to ensure training stability and high performance: Zero-Centered RMSNorm replaces conventional QK-Norm, an Attention Output Gating mechanism eliminates issues like Attention Sink and Massive Activation, and Multi-Token Prediction (MTP) enhances contextual coherence, generation speed, and overall performance.

Qwen3-Next-80B-A3B-Instruct achieves impressive results on key benchmarks, nearly matching the performance of the flagship Qwen3-235B-A22B-Instruct-2507 model despite significantly lower computational costs. On Arena-Hard v2, it scores 82.7 points, outperforming many competing models. In programming, it achieves a solid 56.6 on LiveCodeBench v6, surpassing even some larger models. On the AIME25 math benchmark, it reaches 69.5 points, demonstrating strong capabilities in complex reasoning.

Thanks to its unique architectural innovations and high efficiency, Qwen3-Next-80B-A3B-Instruct is ideally suited for a wide range of applications, including processing extremely long documents, software development and programming, agent-based systems, business process automation—and many more use cases beyond this list.

Announce Date: 11.09.2025
Parameters: 82B
Experts: 512
Activated at inference: 3B
Context: 263K
Layers: 48, using full attention: 12
Attention Type: Hybrid Attention
Mamba Type: Gated Delta Net
Developer: Qwen
Transformers Version: 4.57.0.dev0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore Qwen3-Next-80B-A3B-Instruct capabilities. You can obtain an API access token on the token management page after registration and verification.

An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.

A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.

We recommend creating private endpoints when you need to:

• achieve maximum endpoint performance,
• access the full context window for processing large requests and conversations,
• ensure maximum security by processing data in a dedicated, isolated environment.

Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.