Qwen2.5-7B-1M

Qwen2.5-7B-1M is an advanced compact model built on a state-of-the-art Transformer architecture, integrating key innovations such as: Rotary Positional Embeddings (RoPE) for efficient encoding of positional information, the SwiGLU activation function for nonlinear transformations, RMSNorm with pre-normalization for stable training, and QKV bias in the attention mechanism. The architecture includes 28 layers with Grouped Query Attention (GQA), which enables optimal KV-cache utilization and reduced computational costs.

The main feature of the model is its ability to process up to 1,010,000 tokens of input context! This is equivalent to processing 10 full-length novels, 150 hours of speech transcripts, or 30,000 lines of code within a single request. Dual Chunk Attention (DCA) divides sequences into chunks and redistributes relative positions, ensuring stable performance on ultra-long contexts. Integration with YaRN attention scaling further enhances focus on critical information even when processing extremely long sequences. As a result, the model demonstrates outstanding accuracy in information retrieval tasks from very large documents, achieving over 80% accuracy even with a one-million-token context.

Qwen2.5-7B-1M opens up new possibilities in document processing, automated analysis, and intelligent assistants. It is ideally suited for legal analysis of lengthy contracts and documentation, scientific research, software development involving analysis of large codebases, and building technical support systems with access to extensive knowledge bases. In education, the model can analyze entire textbooks and generate comprehensive learning materials, while in business analytics, it can process voluminous reports and extract key insights for informed decision-making.


Announce Date: 26.01.2025
Parameters: 7.62B
Context: 1M
Layers: 28
Attention Type: Full Attention
VRAM requirements: 60.0 GB using 4 bits quantization
Developer: Qwen
Transformers Version: 4.47.1
License: Apache 2.0

Public endpoint

Use our pre-built public endpoints for free to test inference and explore Qwen2.5-7B-1M capabilities. You can obtain an API access token on the token management page after registration and verification.
Model Name Context Type GPU TPS Status Link
There are no public endpoints for this model yet.

Private server

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

  • maximize endpoint performance,
  • enable full context for long sequences,
  • ensure top-tier security for data processing in an isolated, dedicated environment,
  • use custom weights, such as fine-tuned models or LoRA adapters.

Recommended configurations for hosting Qwen2.5-7B-1M

Prices:
Name vCPU RAM, MB Disk, GB GPU Price, hour
teslaa10-4.16.64.160
1,010,000.0
16 65536 160 4 $1.62 Launch
teslaa2-6.32.128.160
1,010,000.0
32 131072 160 6 $1.65 Launch
rtxa5000-4.16.128.160.nvlink
1,010,000.0
16 131072 160 4 $2.34 Launch
teslaa100-1.16.64.160
1,010,000.0
16 65536 160 1 $2.58 Launch
rtx3090-4.16.64.160
1,010,000.0
16 65536 160 4 $3.10 Launch
teslav100-3.64.256.320
1,010,000.0
64 262144 320 3 $3.89 Launch
rtx4090-4.16.64.160
1,010,000.0
16 65536 160 4 $4.14 Launch
rtx5090-3.16.96.160
1,010,000.0
16 98304 160 3 $4.34 Launch
teslah100-1.16.64.160
1,010,000.0
16 65536 160 1 $5.11 Launch
h200-1.16.128.160
1,010,000.0
16 131072 160 1 $6.98 Launch
Prices:
Name vCPU RAM, MB Disk, GB GPU Price, hour
teslaa10-4.16.64.160
1,010,000.0
16 65536 160 4 $1.62 Launch
teslaa2-6.32.128.160
1,010,000.0
32 131072 160 6 $1.65 Launch
rtxa5000-4.16.128.160.nvlink
1,010,000.0
16 131072 160 4 $2.34 Launch
teslaa100-1.16.64.160
1,010,000.0
16 65536 160 1 $2.58 Launch
rtx3090-4.16.64.160
1,010,000.0
16 65536 160 4 $3.10 Launch
teslav100-3.64.256.320
1,010,000.0
64 262144 320 3 $3.89 Launch
rtx4090-4.16.64.160
1,010,000.0
16 65536 160 4 $4.14 Launch
rtx5090-3.16.96.160
1,010,000.0
16 98304 160 3 $4.34 Launch
teslah100-1.16.64.160
1,010,000.0
16 65536 160 1 $5.11 Launch
h200-1.16.128.160
1,010,000.0
16 131072 160 1 $6.98 Launch
Prices:
Name vCPU RAM, MB Disk, GB GPU Price, hour
teslaa2-6.32.128.160
1,010,000.0
32 131072 160 6 $1.65 Launch
teslaa10-4.16.128.160
1,010,000.0
16 131072 160 4 $1.75 Launch
rtxa5000-4.16.128.160.nvlink
1,010,000.0
16 131072 160 4 $2.34 Launch
teslaa100-1.16.128.160
1,010,000.0
16 131072 160 1 $2.71 Launch
rtx3090-4.16.96.320
1,010,000.0
16 98304 320 4 $3.18 Launch
teslav100-3.64.256.320
1,010,000.0
64 262144 320 3 $3.89 Launch
rtx4090-4.16.96.320
1,010,000.0
16 98304 320 4 $4.22 Launch
rtx5090-3.16.96.160
1,010,000.0
16 98304 160 3 $4.34 Launch
teslah100-1.16.128.160
1,010,000.0
16 131072 160 1 $5.23 Launch
h200-1.16.128.160
1,010,000.0
16 131072 160 1 $6.98 Launch

Related models

QwQ

Need help?

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.