Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Granite Switch 4.1 introduces a unified endpoint that combines natural-language chat, specialized decision-making, and speech-to-text capabilities using a single hosted model. The update leverages a Decision adapter and the fastest ASR model on the OpenASR leaderboard, enabling seamless transitions between tasks without additional GPU memory overhead. By reusing the base model’s KV cache across adapters, the system maintains efficiency while expanding functionality, allowing organizations to deploy more capabilities on existing hardware.

The release includes precompiled Granite Switch 4.1 models in 8B and 30B parameter sizes, bundled with a set of adapters for immediate use. Users can also compose custom models by mixing adapters from Granite Libraries, tailoring workflows to specific needs without starting from scratch. The modular design supports rapid adaptation, with cache-efficient Activated LoRAs and contributions to vLLM further optimizing inference speeds for low-rank adapters.

A new Speech-to-Decisions demo space demonstrates real-time processing, where the 8B model transcribes audio, generates summaries, screens for harmful content, and evaluates 15 user-defined flags—all from one endpoint. The system’s performance was benchmarked against Jev-27B in a five-step RAG workflow, with Granite-Switch-4.1-30B completing the task significantly faster while maintaining accuracy.

Hugging Face plans to expand community involvement by releasing an Adapter SDK for training custom adapters in Mellea, alongside tutorials and Granite Switch support. The framework’s flexibility extends beyond text tasks, as demonstrated by a 3B parameter Granite Switch model capable of playing Doom at 60–100 decisions per second while narrating gameplay, outperforming Jev’s reported 10 decisions per second.