granite-4.0-h-tiny

Granite-4.0-H-Tiny is a compact hybrid Mixture of Experts model with 7 billion total parameters and only 1 billion active parameters during inference. Architecturally, H-Tiny mirrors the structure of H-Small with the same 9:1 ratio of Mamba-2 to Transformer layers but utilizes fewer parameters in each layer. The model was trained on a corpus of 22 trillion tokens, ensuring high quality on enterprise tasks despite its compact size. It supports a context length of up to 128K tokens, with theoretical extensibility thanks to Mamba-2's constant memory requirements.

The performance of H-Tiny is impressive. On the IFEval benchmark, the model achieves a score of 81.44% on average, while on MMLU tasks, it scores 68.65%, demonstrating strong comprehension and reasoning capabilities.

This model is specifically designed for edge deployments, local applications, and low-latency scenarios where response speed and minimal resource requirements are critical. According to the developers, the model requires only 8 GB of memory in 8-bit mode, allowing it to run on consumer-grade GPUs like the RTX 3060 with 12GB of VRAM.

In enterprise scenarios, H-Tiny is recommended as a fast component for executing specific tasks within larger agent systems, as well as in use cases where data privacy compliance is crucial. For example, the model can handle function calling, data extraction and anonymization, or classification, offloading more complex reasoning tasks to other models within the system.

Announce Date: 02.10.2025
Parameters: 6.94B
Experts: 64
Activated at inference: 1B
Context: 131K
Layers: 40, using full attention: 4
Attention Type: Mamba Attention
VRAM requirements: 9.0 GB using 4 bits quantization
Developer: IBM
Transformers Version: 4.56.0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore granite-4.0-h-tiny capabilities. You can obtain an API access token on the token management page after registration and verification.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.