DeepSeek-R1-0528-Qwen3-8B

DeepSeek-R1-0528-Qwen3-8B is a compact 8-billion-parameter model created by distilling knowledge and reasoning capabilities from the flagship DeepSeek-R1-0528 into the Qwen3-8B base model. The model uses an architecture identical to Qwen3-8B, but incorporates the tokenizer from DeepSeek-R1-0528, ensuring compatibility with more advanced reasoning capabilities.

It demonstrates outstanding performance, achieving 86.0% on AIME 2024 — exceeding the base Qwen3-8B by 10% and matching the performance of the much larger Qwen3-235B-Thinking. These results, along with strong benchmark scores across other evaluations, place it among the leading open-source models in its class. The model serves as a great example of a well-implemented distillation process. Reasoning chains from DeepSeek-R1-0528 have been successfully transferred into a more compact architecture, opening new possibilities for academic research and industrial development of small, specialized models. Its compact size of 8B parameters makes it accessible for deployment on less powerful hardware while maintaining high-quality reasoning abilities.

DeepSeek-R1-0528-Qwen3-8B is ideally suited for educational applications, small-scale research projects, and any scenario where a capable reasoning-style answering model is needed, but deploying large reasoning models is not feasible.

Announce Date: 28.05.2025
Parameters: 8.19B
Context: 131K
Attention Type: Full or Sliding Window Attention
VRAM requirements: 21.8 GB using 4 bits quantization
Developer: DeepSeek
Transformers Version: 4.51.0
License: Apache 2.0

Use our pre-built public endpoints to test inference and explore DeepSeek-R1-0528-Qwen3-8B capabilities.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.