Hume + HF 7/15 ship Real World VoiceEQ: voice AI benchmark built on 1M+ human ratings, noise-backed WER is 4x music-backed

On 2026-07-15 Hume and Hugging Face jointly released Real World VoiceEQ — a voice AI quality benchmark built on 1M+ individual human ratings across demographics, speaking styles, and acoustic environments; covers 40+ proprietary and open-source voice models across 15+ evaluation dimensions and 60+ metrics spanning ASR / TTS / S2S / Speech Understanding; the current dataset is 785k TTS ratings and 48k STS ratings; every evaluation ran on Hume's Kairos platform; the post quotes transcription WER on noise-backed speech as roughly 4x higher than on music-backed speech; the public leaderboard is at huggingface.co/spaces/HumeAI/rw-voice-eq and the technical report is at arXiv 2607.14846.