2025
Recent advances in AI music (AIM) generation services are currently transforming the music industry. Given these advances, understanding how humans perceive AIM is crucial both to educate users on identifying AIM songs, and, conversely, to improve current models. This article present results from a listener-focused experiment aimed at understanding how humans perceive AIM. In a blind, Turing-like test, participants were asked to distinguish, from a pair, the AIM and human-made song. This article contrasts with other studies by utilizing a randomized controlled crossover trial that controls for pairwise similarity and allows for a causal interpretation. This study is also the first study to employ a novel, author-uncontrolled dataset of AIM songs from real-world usage of commercial models (i.e., Suno). It was established that listeners' reliability in distinguishing AIM causally increases when pairs are similar. Lastly, through the conduction of a mixed-methods content analysis of listeners' free-form feedback, it is revealed a focus on vocal and technical cues in their judgments.