granite-4.0-h-small

Granite-4.0-H-Small is the flagship model of its family, designed as a hybrid MoE model with 32 billion total parameters and 9 billion active parameters during inference. Architecturally, the model combines Mamba-2 blocks with transformer blocks in a 9:1 ratio. The core of this approach is that Mamba-2 efficiently processes the global context with linear computational complexity, periodically passing information to transformer blocks for more detailed analysis of the local context via the self-attention mechanism. Unlike traditional transformers, where computational cost grows quadratically with increasing sequence length, Mamba-2 scales linearly, and memory requirements remain constant regardless of context length. The model utilizes a fine-grained mixture of experts with shared experts that are constantly active, enhancing parameter efficiency. An important feature is the absence of positional encoding (NoPE), as Mamba inherently preserves token order information thanks to its sequential processing.

On the IFEval benchmark (which measures the ability to follow instructions), the model scores 0.89 points, surpassing all open-source models except for Llama 4 Maverick with 402 billion parameters—a model 12 times larger. The model also shows superior results on MTRAG, a benchmark for complex RAG tasks with multi-turn conversations, unanswerable questions, and information from diverse domains. On the Berkeley Function Calling Leaderboard v3 (BFCL), it demonstrates competitive results with much larger models.

This model is developed as a workhorse for key enterprise tasks such as RAG (Retrieval-Augmented Generation) and agent workflows. It is distributed under the open Apache-2.0 license, complies with international AI safety standards, excels at scaling in terms of context length and batch size, and, importantly, is significantly less resource-intensive than other models of comparable size.

Announce Date: 02.10.2025
Parameters: 33B
Experts: 72
Activated at inference: 9B
Context: 132K
Layers: 40, using full attention: 4
Attention Type: Hybrid Attention
Mamba Type: Mamba 2
Developer: IBM
Transformers Version: 4.56.0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore granite-4.0-h-small capabilities. You can obtain an API access token on the token management page after registration and verification.

An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.

A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.

We recommend creating private endpoints when you need to:

• achieve maximum endpoint performance,
• access the full context window for processing large requests and conversations,
• ensure maximum security by processing data in a dedicated, isolated environment.

Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.