Egyptian and Nigerian English join the Open ASR Leaderboard

Hugging Face’s Open ASR Leaderboard previously included private speech datasets covering five accents from Appen and DataOceanAI, alongside a split for Indian-accented English from Voice Arena. The additions aimed to enhance trustworthy evaluations by incorporating held-out sets to prevent benchmaxxing. Community feedback emphasized the value of these private datasets for more reliable assessments of automatic speech recognition systems. The platform’s focus on expanding language and accent representation has driven continuous updates to the leaderboard.

The latest collaboration with Liva AI introduces two new accents—Egyptian and Nigerian English—bringing the total to eight distinct accents. This expansion aligns with the leaderboard’s goal of improving representation in ASR evaluations by including more accents and languages. The inclusion of these accents enables a richer breakdown based on Kachru’s Three Circles model, providing deeper insights into performance across different linguistic contexts. The updates reflect ongoing efforts to refine evaluation standards in the field.

Liva AI contributes multimodal interaction data collected through real-world consumer products, offering natural, high-context behavior from actual users. Unlike artificially elicited or task-based datasets, Liva’s approach captures authentic interactions across domains, industries, and languages. The company’s ownership of both data collection and distribution environments allows it to design products addressing specific model gaps. This method ensures access to diverse and contextually rich speech data for more robust ASR evaluations.

The Open ASR Leaderboard now reports accent performance using Kachru’s Three Circles framework, categorizing accents into Inner, Outer, and Expanding Circles. As additional accents are added, the Outer and Expanding Circles will be further subdivided to reflect linguistic diversity. The leaderboard’s structure supports a more granular analysis of ASR performance across global English variants. This initiative underscores the platform’s commitment to fostering inclusive and trustworthy speech recognition benchmarks.