Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms

Apple introduced AnySpeechRecognizer and AnyDecisionModel, two new Swift packages that allow developers to integrate open speech recognition and decision models into apps on Apple platforms. The packages mirror Apple’s SpeechAnalyzer API, enabling developers to switch between Apple’s default engine and third-party models like NVIDIA’s Parakeet or Alibaba’s Qwen3-ASR without rewriting core logic. Models are downloaded from the Hugging Face Hub and run locally, with downloads triggered only when requested by the app.

AnySpeechRecognizer adds speaker diarization, a feature missing from Apple’s Speech framework, using NVIDIA’s Streaming Sortformer to attribute speech to up to four speakers in real time. The package supports both file-based and live audio input, with performance varying by model: Parakeet processes an hour of audio in 20–40 seconds from a file, while Qwen3-ASR is slower but supports more languages and dialects. The package includes a command-line tool, hear, for evaluating performance on custom datasets.

AnyDecisionModel extends the same approach to decision-making tasks, allowing developers to choose between open models and Apple’s built-in tools. Both packages are designed to work with the existing AnyLanguageModel and swift-huggingface infrastructure, maintaining compatibility with Swift’s async sequences and Hugging Face’s Core ML runtime. The packages are available now on GitHub, with documentation and examples provided in their repositories.

The new packages are part of Apple’s ongoing effort to expand on-device AI capabilities using open models. They leverage Core ML for local execution, with models converted from their original checkpoints and pinned to specific revisions for reproducibility. The packages also support sandboxed and non-sandboxed Mac apps, sharing downloaded models with other Hugging Face tools via the standard cache directory.