Meta is back with Muse Glimmer: local, agentic, multimodal, and open source

Meta’s Muse Glimmer, a new 30-billion-parameter multimodal model, was released today under the Apache 2.0 license, enabling local deployment for privacy-sensitive or cost-effective use cases. The model is optimized for agentic applications, including coding assistants, document analysis, and personal assistants, and supports both images and videos. Meta provides day-zero integration with libraries such as transformers, llama.cpp, vLLM, and Inference Endpoints, facilitating immediate adoption and experimentation.

Muse Glimmer features a 2-billion-parameter vision encoder, derived from Meta’s Perception Encoder architecture, which processes images and videos into embeddings for the text decoder. The encoder uses a patch-based approach with 2D RoPE attention and reduces image tokens via pixel shuffle before projection. Videos are processed frame-by-frame at 2 frames per second, capped at 96 frames, with timestamped placeholders for multimodal input. The model supports text-only, image-text, and video inference, including multimodal tool calling and object detection.

The model is compatible with NVIDIA, AMD, and Intel GPUs, and can be loaded using AutoModelForMultimodalLM and AutoProcessor classes in transformers. Meta recommends installing torchvision for image-text tasks and torchcodec for video processing. Muse Glimmer demonstrates strong performance on video question-answering benchmarks such as VideoMME2, and supports structured content generation through speculative decoding with DFlash.

Muse Glimmer includes day-zero support for llama.cpp, with calibrated quantized versions available from Meta and Unsloth. The model supports speculative decoding via DFlash, which accelerates generation at the cost of additional memory. Users can deploy the model locally using llama.cpp’s server or CLI, with a built-in WebUI accessible at localhost:8080, or integrate it with coding agents like Pi.