granite-4.0-h-tiny

Granite-4.0-H-Tiny is a compact hybrid Mixture of Experts model with 7 billion total parameters and only 1 billion active parameters during inference. Architecturally, H-Tiny mirrors the structure of H-Small with the same 9:1 ratio of Mamba-2 to Transformer layers but utilizes fewer parameters in each layer. The model was trained on a corpus of 22 trillion tokens, ensuring high quality on enterprise tasks despite its compact size. It supports a context length of up to 128K tokens, with theoretical extensibility thanks to Mamba-2's constant memory requirements.

The performance of H-Tiny is impressive. On the IFEval benchmark, the model achieves a score of 81.44% on average, while on MMLU tasks, it scores 68.65%, demonstrating strong comprehension and reasoning capabilities.

This model is specifically designed for edge deployments, local applications, and low-latency scenarios where response speed and minimal resource requirements are critical. According to the developers, the model requires only 8 GB of memory in 8-bit mode, allowing it to run on consumer-grade GPUs like the RTX 3060 with 12GB of VRAM.

In enterprise scenarios, H-Tiny is recommended as a fast component for executing specific tasks within larger agent systems, as well as in use cases where data privacy compliance is crucial. For example, the model can handle function calling, data extraction and anonymization, or classification, offloading more complex reasoning tasks to other models within the system.

Announce Date: 02.10.2025
Parameters: 7B
Experts: 64
Activated at inference: 1B
Context: 132K
Layers: 40, using full attention: 4
Attention Type: Hybrid Attention
Mamba Type: Mamba 2
Developer: IBM
Transformers Version: 4.56.0
License: Apache 2.0

Use our pre-built public endpoints for free to test inference and explore granite-4.0-h-tiny capabilities. You can obtain an API access token on the token management page after registration and verification.

An endpoint is a server with pre-installed software that automatically downloads neural network weights and launches an OpenAI-compatible API to handle incoming requests.

A private endpoint is created within the user's project, belongs exclusively to the user, and is billed per server configuration in accordance with the infrastructure pricing.

We recommend creating private endpoints when you need to:

• achieve maximum endpoint performance,
• access the full context window for processing large requests and conversations,
• ensure maximum security by processing data in a dedicated, isolated environment.

Contact our dedicated neural networks support team at support@immers.cloud or send your request to the sales department at sale@immers.cloud.