Meta Llama 3.1 8B is an impressive and incredibly popular model within the community, featuring 8 billion parameters. It is part of the innovative Llama 3.1 series, which demonstrates revolutionary capabilities in the fields of natural language processing and intelligent dialogue. Architecturally, the model is built on an optimized transformer architecture using advanced Grouped-Query Attention (GQA) technology, providing exceptional scalability for inference. The model was trained on more than 15 trillion tokens from publicly available sources, enabling it to demonstrate deep contextual understanding and generate high-quality responses.
Officially, the model supports a context window of up to 128 thousand tokens and works with eight languages (Russian is not listed, although the model understands it very well). The instruction-tuned version of the model underwent supervised fine-tuning (SFT) and reinforcement learning based on human feedback (RLHF), ensuring a high level of safety and alignment with human preferences.
The potential applications of the model are truly **boundless**: from building intelligent dialogue systems and personal assistants to developing tools for programming and processing large volumes of text. The model supports native function calling (tool calling), making it possible to integrate with external services and create powerful agent-based systems.Thanks to its permissive Llama 3.1 Community License, the model is available for both commercial and research use, making Llama 3.1 8B an ideal choice for developers, researchers, and businesses looking to implement cutting-edge AI technologies into their products and services.
| Model Name | Context | Type | GPU | Status | Link |
|---|
There are no public endpoints for this model yet.
An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.
A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.
We recommend creating private endpoints when you need to:
| Name | GPU | TPS | Max Concurrency | |||
|---|---|---|---|---|---|---|
131,072.0 tensor |
2 | $0.57 | 55.16 | 1.167 | Launch | |
131,072.0 tensor |
2 | $0.93 | 105.94 | 1.988 | Launch | |
131,072.0 tensor |
2 | $1.56 | 170.94 | 2.147 | Launch | |
131,072.0 tensor |
2 | $1.56 | 2.147 | Launch | ||
131,072.0 |
1 | $1.59 | 124.49 | 1.336 | Launch | |
131,072.0 tensor |
4 | $1.82 | 192.97 | 1.367 | Launch | |
131,072.0 tensor |
2 | $1.92 | 209.47 | 2.141 | Launch | |
131,072.0 |
1 | $2.37 | 172.11 | 4.080 | Launch | |
131,072.0 |
1 | $3.83 | 187.27 | 4.032 | Launch | |
131,072.0 |
1 | $4.11 | 227.40 | 4.875 | Launch | |
131,072.0 tensor |
2 | $4.61 | 8.514 | Launch | ||
131,072.0 |
1 | $4.74 | 7.561 | Launch | ||
131,072.0 tensor |
2 | $9.40 | 15.478 | Launch | ||
| Name | GPU | TPS | Max Concurrency | |||
|---|---|---|---|---|---|---|
131,072.0 tensor |
2 | $0.93 | 1.878 | Launch | ||
131,072.0 pipeline |
3 | $1.06 | 20.04 | 1.654 | Launch | |
131,072.0 tensor |
4 | $1.26 | 2.475 | Launch | ||
131,072.0 tensor |
2 | $1.56 | 129.07 | 2.000 | Launch | |
131,072.0 tensor |
2 | $1.56 | 2.000 | Launch | ||
131,072.0 |
1 | $1.59 | 1.181 | Launch | ||
131,072.0 tensor |
4 | $1.82 | 1.327 | Launch | ||
131,072.0 tensor |
2 | $1.92 | 129.37 | 1.996 | Launch | |
131,072.0 |
1 | $2.37 | 131.49 | 3.935 | Launch | |
131,072.0 |
1 | $3.83 | 165.97 | 3.931 | Launch | |
131,072.0 |
1 | $4.11 | 4.731 | Launch | ||
131,072.0 tensor |
2 | $4.61 | 8.405 | Launch | ||
131,072.0 |
1 | $4.74 | 7.416 | Launch | ||
131,072.0 tensor |
2 | $9.40 | 15.368 | Launch | ||
| Name | GPU | TPS | Max Concurrency | |||
|---|---|---|---|---|---|---|
131,072.0 tensor |
2 | $0.93 | 53.99 | 1.478 | Launch | |
131,072.0 pipeline |
3 | $1.06 | 1.254 | Launch | ||
131,072.0 tensor |
4 | $1.26 | 2.077 | Launch | ||
131,072.0 tensor |
2 | $1.56 | 73.37 | 1.599 | Launch | |
131,072.0 tensor |
2 | $1.56 | 1.599 | Launch | ||
131,072.0 tensor |
2 | $1.92 | 1.595 | Launch | ||
131,072.0 |
1 | $2.37 | 85.96 | 3.533 | Launch | |
131,072.0 tensor |
2 | $2.93 | 2.495 | Launch | ||
131,072.0 |
1 | $3.83 | 101.48 | 3.528 | Launch | |
131,072.0 |
1 | $4.11 | 4.328 | Launch | ||
131,072.0 tensor |
2 | $4.61 | 8.004 | Launch | ||
131,072.0 |
1 | $4.74 | 7.014 | Launch | ||
131,072.0 tensor |
2 | $9.40 | 14.967 | Launch | ||
Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.