How Arm is helping developers build local AI systems for the intelligent edge

Arm describes the shift toward agentic AI systems that reason across inputs, select models, and trigger actions in real time, requiring developers to integrate CPUs, GPUs, sensors, and cloud services into cohesive workflows. The company highlights practical challenges such as latency, privacy, power constraints, and network reliability, emphasizing a distributed computing approach rather than a binary choice between edge and cloud. Developers must dynamically decide where each component runs to balance responsiveness, efficiency, and user experience.

At the Arm Create Builder Zone during We Are Developers, attendees explored on-device generative AI and connected applications to Arm-powered devices, with one live translation demo using over 30 models across multiple stages to deliver a responsive experience. The demonstration showcased how different parts of the pipeline used varied frameworks and numerical precisions, such as FP32, FP16, INT8, and INT4, depending on compute or memory constraints for each stage.

A real-time speech generation example, Qwen3-TTS CustomVoice, ran on a single Arm CPU core using Arm Scalable Matrix Extension 2 (SME2), illustrating how advanced workloads can now execute locally without relying on cloud round trips. Another demonstration paired the Reachy Mini robot with NVIDIA DGX Spark to show how perception, conversation, and movement can be coordinated in a single local loop, treating AI tasks as interconnected components of a responsive system.

Arm introduces the AI Portal to help developers navigate the complexity of building agentic AI systems, offering pre-optimized models, performance evaluations, and deployment workflows tailored for Arm-based targets. The portal supports trade-off assessments, including model size, platform compatibility, quality, and expected performance, while also providing access to code and tools to streamline model selection to implementation. Additionally, the Arm Model Context Protocol (MCP) Server integrates relevant resources directly into developer workflows, enabling coding agents to access Arm tools for model discovery, evaluation, optimization, and deployment.