Llama-3-8B-Instruct

The Llama-3-8B model is rightfully considered one of the most famous and popular models in the history of open-source artificial intelligence. Released on April 18, 2024, it marked a turning point, making technologies used in cutting-edge AI models accessible to a wide community of researchers and developers. It was Llama 3 that became the catalyst for the explosive growth of open-source projects and startups in the AI field, proving that open models can successfully compete with commercial counterparts, not only in terms of customization flexibility and fine-tuning but also in terms of performance quality.

At the core of the model lies a transformer architecture utilizing the Grouped-Query Attention (GQA) mechanism. Unlike standard MHA (Multi-Head Attention), GQA significantly reduces memory load and accelerates generation. The model works with a context of 8,192 tokens and uses a tokenizer with a vocabulary of 128,256 tokens, allowing it to efficiently process complex multilingual queries.

The uniqueness of Llama-3-8B is largely due to the unprecedented scale and quality of its training at the time of its release. The base model was pre-trained on over 15 trillion tokens — seven times more than its predecessor, Llama 2. The data, collected from publicly available sources, was carefully filtered to ensure high quality. To obtain the instruction-tuned version (-Instruct), a two-stage procedure was used: first, Supervised Fine-Tuning (SFT) on millions of examples, followed by alignment with human preferences using Reinforcement Learning from Human Feedback (RLHF). This made the model not only extremely helpful but also significantly reduced the number of false refusals.

Thanks to its characteristics, Llama-3-8B still has a wide range of practical applications to this day. A key property is the ease of its fine-tuning and adaptation. This feature allows the model to be used both as a foundation for creating dialogue systems and text analysis tools, and as an ideal "sandbox" for research experiments with fine-tuning for highly specialized domains such as law or medicine. Moreover, the combination of an open commercial license, low hardware requirements, and ease of customization made it the number one choice for startups and rapid prototyping, enabling the efficient creation of MVPs in a wide variety of subject areas.

Announce Date: 17.04.2024
Parameters: 9B
Context: 9K
Layers: 32
Attention Type: Full Attention
Developer: Meta AI
Transformers Version: 4.40.0.dev0
License: META LLAMA 3

Use our pre-built public endpoints for free to test inference and explore Llama-3-8B-Instruct capabilities. You can obtain an API access token on the token management page after registration and verification.

An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.

A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.

We recommend creating private endpoints when you need to:

• achieve maximum endpoint performance,
• access the full context window for processing large requests and conversations,
• ensure maximum security by processing data in a dedicated, isolated environment.

Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.