Phi-4-multimodal

Phi-4-multimodal-instruct is an open-source multimodal model from Microsoft that processes text, images, and audio in a unified architectural solution. It builds upon the Phi-3.5 and Phi-4.0 technologies, featuring an extended context window of 128K tokens and support for 23 languages in text (including Russian), 8 languages in audio, and English for visual tasks. The model is optimized for environments with limited computational resources and low-latency scenarios, demonstrating strong performance in mathematics, logic, speech recognition, translation, and image analysis.  

A single neural network handles text, images (OCR, tables, diagrams), and audio (recognition, translation, summarization). For example, in DocVQA benchmarks, the model achieves 93.2% accuracy, outperforming Gemini-2.0-Flash (92.1%).  

The model is ideal for multisensory applications—joint processing of audio and images (e.g., video analysis with subtitles). At the same time, thanks to optimization via Microsoft Olive and ONNX GenAI Runtime, it can be deployed on edge devices, including smartphones and IoT systems, even with limited computational resources.

Announce Date: 27.02.2025
Parameters: 5.57B
Context: 131K
Attention Type: Full Attention
VRAM requirements: 18.6 GB using 4 bits quantization
Developer: Microsoft
Transformers Version: 4.46.1
License: MIT

Use our pre-built public endpoints to test inference and explore Phi-4-multimodal capabilities.

Rent your own physically dedicated instance with hourly or long-term monthly billing.

We recommend deploying private instances in the following scenarios:

• maximize endpoint performance,
• enable full context for long sequences,
• ensure top-tier security for data processing in an isolated, dedicated environment,
• use custom weights, such as fine-tuned models or LoRA adapters.

Contact our dedicated neural networks support team at nn@immers.cloud or send your request to the sales department at sale@immers.cloud.