Falcon-Emirati: When an LLM Learns the Dialect, the Culture, and the Nuance

Arabic encompasses diverse dialects, with Emirati Arabic differing significantly from Modern Standard Arabic in vocabulary, rhythm, and cultural context. A model trained only on MSA may translate words accurately but miss the intended meaning entirely. Falcon-Emirati-7B is designed to bridge this gap by specializing in Emirati Arabic, capturing its unique expressions, tone, and cultural references. The model builds on the Falcon-H1-Arabic family, which combines State Space Models and Transformer attention for efficiency and precision in long-sequence processing.

The team constructed a dedicated data pipeline for Emirati dialect adaptation, sourcing authentic Emirati web content and MSA materials about Emirati culture, heritage, and social norms. Synthetic Emirati-dialect data was generated under strict linguistic rules to ensure authenticity, addressing gaps in real-world conversational coverage. Training strategies were experimentally refined, balancing authentic and synthetic data while incorporating cultural context to avoid superficial fluency.

Evaluation relied on both native-speaker manual reviews and the Alyah benchmark, a 1,173-sample multiple-choice test designed to assess Emirati dialect competence. Native speakers judged outputs for naturalness, tone, and cultural appropriateness, while Alyah measured performance across categories like greetings, heritage, and poetry. The model achieved 84.83% accuracy on Alyah, outperforming larger multilingual and Arabic-focused models in the comparison.

Falcon-Emirati-7B is built on the 7B-parameter variant of Falcon-H1-Arabic, balancing quality and practicality for dialect specialization. The model targets Emirati vocabulary, grammar, and cultural knowledge that general Arabic models often overlook, aiming to reflect native speakers' usage in real contexts.