An efficient natively multimodal MoE model with a context window of up to 1 million tokens, advancing the M-3 lineup and designed for programming, agentic workflows, and cybersecurity. Key architectural improvements over the previous version include the use of sparse attention in all layers, the use of mxfp8 quantization during training, and DSpark speculative decoding. Together, these innovations provide high inference speed while maintaining quality and reducing infrastructure requirements for inference.
DeepSeek-V4.1-Flash-1017 is a focused and highly efficient update to the previous version, DeepSeek-V4.1-Flash. Its architectural blocks are unchanged: it is a multimodal MoE model with 552B parameters and a context window of up to 1M tokens, built on a combination of a Causal Encoder-Decoder and the Compressed Sparse Attention 2 mechanism. The central idea of this release is a substantial improvement in performance on logical reasoning, programming, and agentic interaction tasks while simultaneously reducing task execution time and the length of the generated response. The model is aimed at long agentic loops, software development, office automation, and multimodal visual scenarios.
DeepSeek-V4.1-Flash is a 552B (+196.6B Engram parameters and 14.2B DSpark MTP parameters) multimodal MoE model with context up to 1M tokens, built on an asymmetric Causal Encoder-Decoder architecture and CSA2. It reduces the global KV cache to 890 bytes per token and activates 8B/16B parameters on prefill/decode, delivering strong quality in agentic and multimodal tasks at substantially lower memory costs on long context.
The first open multimodal model from DeepSeek AI built on the MoE architecture (305B parameters, 13B active), equipped with a ViT visual encoder, CSA+HCA attention mechanisms for efficient processing of 1M-token context, and DSpark for accelerating generation. The model is designed for agentic scenarios and workflow automation where image analysis and the ability to work with extremely long context without additional integrations are important.
The first natively multimodal model in the GLM-5 series, previously known as ox-alpha. The model is also the first to use a hybrid architecture (sparse + linear attention), with 320B total / 18B active parameters. Despite its relatively moderate size, it delivers flagship-level performance, coming very close to Claude Opus 4.8 in quality, while being distributed under the MIT license. The model supports context up to 1M tokens, image and video processing, and a controllable reasoning budget via the reasoning_effort parameter.
Flagship model from Z.ai with 744 billion parameters (40 billion active) and a context length of 1 million tokens, built on a MoE + DeepSeek Sparse Attention architecture with IndexShare technology. The model retains the GLM-5.2 design but achieves a substantial quality improvement through post-training scaling, reaching SOTA among open-source models in programming and demonstrating the best cybersecurity capabilities among both open and closed models, taking first place on CyberGym with a score of 84.5%.
An experimental multimodal MoE model with 125 billion parameters (6 billion activated per token), an architectural precursor to Qwen4. Key innovations include: a hybrid of Gated DeltaNet and Qwen Sparse Attention (QSA) for efficient long-context processing, Gated Residual with 4 parallel branches, a 51-billion-parameter host-side N-gram embedding, and the Muon optimizer. The model leads benchmarks among open models in its class.
The official release of the flagship MoE model DeepSeek-V4-Pro with 1.6 trillion parameters (49 billion activated), supporting a 1-million-token context. Thanks to its hybrid CSA + HCA attention architecture, the model achieves record efficiency on long contexts and demonstrates results at the level of top proprietary models in agentic tasks and programming. Compared with the preview version, it features improved agentic capabilities and is optimized for production environments.
The first open Max-level model from Qwen with 2.4 trillion parameters (95 billion activated), using a hybrid Gated DeltaNet and Gated Attention architecture, along with MoE of 512 experts. The model confidently leads or competes with closed models across a range of key benchmarks, supports context up to 262K tokens, and is one of the largest and most powerful open LLMs.
The flagship dense 30B model of the Granite-4.2 family with a native reasoning mode and three operating modes—full thinking, no thinking, and low-effort. Its architecture combines Grouped Query Attention with full attention and a context of up to 128K tokens. The model underwent full agentic RL for software engineering, terminal work, and web search, so it is suitable for autonomous agentic workflows and programming-related tasks. It is recommended for the most demanding systems where quality matters more than latency and inference cost.
A universal dense 8B model with native reasoning, three thinking modes, and native tool calling. Thanks to GQA with full attention and a context of up to 128K, it is capable of working with fairly long documents and multi-step agentic scenarios while remaining practical in terms of deployment. The model underwent agentic RL for SWE, terminal, and search, so it can not only follow instructions but also act in real environments. This is the optimal choice for less complex mid-scale agentic systems where both quality and inference cost matter.
A compact dense 3B model with native reasoning and three thinking modes, aimed at efficient deployment in environments with limited computational resources. It uses the same architectural scheme with GQA and full attention in a context of up to 128K. Unlike the 8B and 30B models, it is not trained with agentic RL for SWE, terminal and search, so it has different strengths and use cases — dialogue, code completion, information extraction and document summarization.
A next-generation compact multimodal model with 27B parameters, built on the Qwen3.5 architecture and enhanced with advanced post-training. The model demonstrates significant progress in programming, agentic tasks, and professional scenarios, outperforming its predecessor Qwen3.6-27B on key benchmarks. Its built-in vision encoder and context expandable to 1M tokens make it a versatile solution for a wide range of tasks — from software development to multimodal automation.
NVIDIA-Nemotron-3.5-Lightning-30B-A3B is a hybrid MoE model (Mamba-2 + MoE + Attention) with 30B total and 3B active parameters, with context expandable to 1M tokens and the ability to integrate speculative decoding support. The NVFP4 variant provides fast inference on GPUs from Ampere to Blackwell, while BF16 is intended for customization and fine-tuning. The model leads the 30B MoE segment in efficiency for always-on agents, significantly outperforming the previous version in the Nemotron-3-Nano line on agentic and coding benchmarks.
An updated version of the popular DeepSeek-V4-Flash model. The new LLM retains the same architecture — it is an MoE model with 284B parameters (13B activated per token), supporting a context of up to 1 million tokens and equipped with a hybrid CSA/HCA attention mechanism for efficient long-sequence processing. Version 0731 adds the DSpark speculative decoding module and substantially strengthens agentic capabilities, enabling the model to compete confidently with larger solutions on key benchmarks while remaining fast, cost-efficient, and fully open for use.
The world's first open-weight 3T-class model (a 2.8 trillion-parameter MoE with 104 billion parameters activated per token), featuring native multimodality and a context window of up to 1 million tokens. Built on a hybrid Kimi Delta Attention (KDA) architecture with the Attention Residuals (AttnRes) mechanism and Stable LatentMoE technique, it delivers frontier-level reasoning, programming, and agentic capabilities, with fully open weights for research and deployment.
A state-of-the-art end-to-end model from Baidu for OCR and document parsing, capable of transcribing dozens of pages in a single forward pass thanks to an innovative Reference Sliding Window Attention (R‑SWA) mechanism that maintains a fixed KV‑cache size and stable generation speed. The model achieves state-of-the-art results on the OmniDocBench benchmark (93.92%), outperforming the baseline DeepSeek OCR by 6% in overall accuracy and by 35% in generation speed.
An open MoE model from Z.ai with 753B parameters, only 39 billion of which are activated per token, optimized for long‑horizon agentic tasks with a context length of one million tokens. Through the innovative IndexShare technique and strong reasoning and coding capabilities, it reaches the level of the best closed models, setting a new standard in the open‑source class.
An open agentic model from Moonshot AI based on a MoE architecture (1T parameters, 32B activated) with MLA attention, native INT4 quantization, and multimodality (text, images, video). The model is optimized for long-horizon tasks, reduces thinking token consumption by 30% compared to K2.6, and competes with leading proprietary solutions.
A flagship language model with 550 billion total parameters (55 billion active per inference pass), built on a hybrid LatentMoE architecture (Mamba-2 + Mixture of Experts + Attention), supporting up to 1 million tokens of context and a switchable reasoning mode. The model achieves record-breaking inference throughput — up to ~6× higher than comparable open LLMs — while matching the accuracy of the best global counterparts. This makes it the ideal choice for complex agentic tasks, long-context analysis, and high-load enterprise-grade scenarios.