granite-4.0-h-small

Granite-4.0-H-Small is the flagship model of its family, designed as a hybrid MoE model with 32 billion total parameters and 9 billion active parameters during inference. Architecturally, the model combines Mamba-2 blocks with transformer blocks in a 9:1 ratio. The core of this approach is that Mamba-2 efficiently processes the global context with linear computational complexity, periodically passing information to transformer blocks for more detailed analysis of the local context via the self-attention mechanism. Unlike traditional transformers, where computational cost grows quadratically with increasing sequence length, Mamba-2 scales linearly, and memory requirements remain constant regardless of context length. The model utilizes a fine-grained mixture of experts with shared experts that are constantly active, enhancing parameter efficiency. An important feature is the absence of positional encoding (NoPE), as Mamba inherently preserves token order information thanks to its sequential processing.

On the IFEval benchmark (which measures the ability to follow instructions), the model scores 0.89 points, surpassing all open-source models except for Llama 4 Maverick with 402 billion parameters—a model 12 times larger. The model also shows superior results on MTRAG, a benchmark for complex RAG tasks with multi-turn conversations, unanswerable questions, and information from diverse domains. On the Berkeley Function Calling Leaderboard v3 (BFCL), it demonstrates competitive results with much larger models.

This model is developed as a workhorse for key enterprise tasks such as RAG (Retrieval-Augmented Generation) and agent workflows. It is distributed under the open Apache-2.0 license, complies with international AI safety standards, excels at scaling in terms of context length and batch size, and, importantly, is significantly less resource-intensive than other models of comparable size.

Announce Date: 02.10.2025
Parameters: 32.2B
Experts: 72
Activated at inference: 9B
Context: 131K
Layers: 40, using full attention: 4
Attention Type: Mamba Attention
VRAM requirements: 30.7 GB using 4 bits quantization
Developer: IBM
Transformers Version: 4.56.0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore granite-4.0-h-small capabilities. You can obtain an API access token on the token management page after registration and verification.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.