granite-4.0-micro

Granite-4.0-Micro is a traditional dense model based on the Transformer architecture with a self-attention mechanism. Unlike the hybrid variants in the family, this model relies entirely on a proven architecture similar to previous Granite generations. The decision to create a traditional version was driven by the need to ensure compatibility with infrastructure and tools where support for Mamba-2 is not yet optimized, such as some versions of llama.cpp, PEFT, and other fine-tuning frameworks. The model uses standard Rotary Positional Encoding (RoPE) to encode token positions, ensuring predictable behavior across various sequence lengths. All 3 billion parameters are active during every forward pass, providing stable and consistent generation quality.

Despite using a traditional Transformer architecture, the model demonstrates a significant quality improvement over previous generations, thanks to new and enhanced training and post-training methodologies, as well as the expansion and refinement of the Granite training data corpus. The model was trained on an extended and carefully curated corpus of 22 trillion tokens, including data from DataComp-LM, GneissWeb, TxT360, Wikipedia, and other corporate-oriented sources. Improved pre-training and post-training techniques ensure superior performance on tasks critical for corporate use, including instruction following, mathematical reasoning, code manipulation, and multilingual capabilities. Post-training utilizes both synthetic and open datasets covering language, code, mathematics, function calling, RAG, and cybersecurity. All training data was prepared using the open-source Data Prep Kit framework.

In terms of performance, Granite-4.0-Micro shows strong results on key benchmarks. On MMLU, the model achieves 65.98%, a competitive result for a model of this size. On IFEval tasks, the model demonstrates an average score of 82.31%, confirming its high instruction-following capabilities. The model is also effective in coding tasks, supports Fill-In-the-Middle (FIM) for code autocompletion, and shows good results in comprehension and generation tasks across various programming languages.

According to the release data, Granite-4.0-Micro requires 9 GB of memory when deployed in 8-bit format with a context of 128K tokens and a batch size of 1. The model can run on consumer-grade GPUs like the RTX 3060 12GB, making it accessible to a wide range of developers. Full compatibility with Hugging Face Transformers, vLLM, llama.cpp, MLX, and other popular inference frameworks ensures easy integration into existing pipelines. The model is particularly suited for scenarios requiring the use of PEFT fine-tuning methods, such as LoRA or QLoRA, where support for hybrid architectures is still under development.

Announce Date: 02.10.2025
Parameters: 3.4B
Context: 131K
Layers: 40
Attention Type: Full Attention
VRAM requirements: 15.1 GB using 4 bits quantization
Developer: IBM
Transformers Version: 4.56.0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore granite-4.0-micro capabilities. You can obtain an API access token on the token management page after registration and verification.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.