Believe in the Sound You See: The Effects of Body Type and Voice Pitch on the Perceived Audio-Visual Correspondence and Believability of Virtual Characters

Lam, Luchcha. M.S., Purdue University, May 2023. Believe in the Sound You See: The Effects of Body Type and Voice Pitch on the Perceived Audio-Visual Correspondence and Believability of Virtual Characters. Major Professor: Nicoletta Adamo. 

We examined the effects of body type and voice pitch of virtual characters’ on perceived audio-visual correspondence and believability. For our within-group study, we developed nine experimental conditions using a 3 (body type: ectomorph vs. mesomorph vs. endo- morph body types) × 3 (voice pitch: low vs. medium vs. high fundamental frequency [F0]). We found significant main effects from voice pitch and significant interaction effects between a character’s body type and voice pitch on both the level of perceived audio-visual correspondence and believability of female and male characters. For female characters, we also observed an additional significant main effect from body type and a significant interac- tion effect between the participant’s biological sex and the character’s voice pitch on both perceived audio-visual correspondence and believability. Moreover, the results show that perceived believability is highly correlated to perceived audio-visual correspondence. Our findings have important practical implications in applications where the character is meant to be an emotional or informational guide that requires some level of believability, as the findings suggest that it is possible to enhance the believability of the characters by generating appropriate voices through pitch manipulation of existing voices.