GigaChat 3 Ultra Preview is a massive MoE model with 702 billion total parameters, of which only 36 billion are active in generating each token. A key feature is the use of Multi-head Latent Attention (MLA) — a mechanism that compresses the Key-Value (KV) cache into a latent vector. For users, this means the ability to work with massive contexts and long documents without an exponential increase in GPU memory requirements. The model is trained using a Multi-Token Prediction (MTP) objective — unlike classical solutions that predict only the next word, this model can predict multiple tokens in a single forward pass. This implements a native speculative decoding mechanism, accelerating response generation by up to 40% without quality loss. Technically, this makes it one of the fastest models in the "heavy" LLM class.
Unlike many Russian models that are the result of fine-tuning foreign counterparts, GigaChat 3 was trained from scratch on a unique dataset of over 20 trillion tokens. This dataset includes a vast amount of Russian-language sources and adds languages rarely featured in model training — from Chinese and Arabic to Uzbek and Kazakh. This provides the model with a deep understanding of the Russian language and Russian cultural context, which are inaccessible to foreign LLMs. The model leads in both Russian and international benchmarks: MERA (Industrial) - 0.824, HumanEval+ (Code) - 0.8659, GSM8K (Math) - 0.9598, etc. In terms of quality, GigaChat 3 Ultra confidently outperforms GigaChat 2 Max on all key benchmarks.
The model is optimized for On-premise and Private Cloud installations in large enterprises where data security is critical (air-gapped environments without internet access). It is ideally suited for complex analytics of large document arrays (RAG), automation of L2/L3 technical support (requiring deep contextual understanding), programmer assistants, and code generation within closed corporate repositories. The model supports popular inference engines (vLLM, SGLang, LMDeploy, TensorRT-LLM) and operates in BF16 and FP8 modes for optimal performance.
| Model Name | Context | Type | GPU | Status | Link |
|---|
There are no public endpoints for this model yet.
An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.
A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.
We recommend creating private endpoints when you need to:
Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.