When two become one: Temporally dynamic integration of the face and voice

Jonathan Freeman, Nalini Ambady

Research output: Contribution to journalArticle

Abstract

In everyday interactions with others, people have to deal with the sight of a face and sound of a voice at the same time. How the perceptual system brings this information together over hundreds of milliseconds to perceive others remains unclear. In 2 studies, we investigated how facial and vocal cues are integrated during real-time social categorization by recording participants' hand movements (via the streaming x, y coordinates of the computer mouse) en route to "male" and "female" responses on the screen. Participants were presented with male and female faces that were accompanied by a same-sex voice morphed to be either sex-typical (e.g., masculinized male voice) or sex-atypical (i.e., feminized male voice). Before settling into ultimate sex categorizations of the face, the simultaneous processing of a sex-atypical voice led the hand to be continuously attracted to the opposite sex-category response across construal. This is evidence that ongoing results from voice perception continuously influence face perception across processing. Thus, social categorization involves dynamic updates of gradual integration of the face and voice.

Original languageEnglish (US)
Pages (from-to)259-263
Number of pages5
JournalJournal of Experimental Social Psychology
Volume47
Issue number1
DOIs
StatePublished - Jan 2011

Fingerprint

settling
recording
interaction
evidence
Hand
Information Systems
Cues
time

Keywords

  • Dynamics
  • Face perception
  • Motor processes
  • Person perception
  • Social categorization
  • Temporal
  • Voice perception

ASJC Scopus subject areas

  • Social Psychology
  • Sociology and Political Science

Cite this

When two become one : Temporally dynamic integration of the face and voice. / Freeman, Jonathan; Ambady, Nalini.

In: Journal of Experimental Social Psychology, Vol. 47, No. 1, 01.2011, p. 259-263.

Research output: Contribution to journalArticle

@article{79949f52616a4dbd8a1efad6fa3fc227,
title = "When two become one: Temporally dynamic integration of the face and voice",
abstract = "In everyday interactions with others, people have to deal with the sight of a face and sound of a voice at the same time. How the perceptual system brings this information together over hundreds of milliseconds to perceive others remains unclear. In 2 studies, we investigated how facial and vocal cues are integrated during real-time social categorization by recording participants' hand movements (via the streaming x, y coordinates of the computer mouse) en route to {"}male{"} and {"}female{"} responses on the screen. Participants were presented with male and female faces that were accompanied by a same-sex voice morphed to be either sex-typical (e.g., masculinized male voice) or sex-atypical (i.e., feminized male voice). Before settling into ultimate sex categorizations of the face, the simultaneous processing of a sex-atypical voice led the hand to be continuously attracted to the opposite sex-category response across construal. This is evidence that ongoing results from voice perception continuously influence face perception across processing. Thus, social categorization involves dynamic updates of gradual integration of the face and voice.",
keywords = "Dynamics, Face perception, Motor processes, Person perception, Social categorization, Temporal, Voice perception",
author = "Jonathan Freeman and Nalini Ambady",
year = "2011",
month = "1",
doi = "10.1016/j.jesp.2010.08.018",
language = "English (US)",
volume = "47",
pages = "259--263",
journal = "Journal of Experimental Social Psychology",
issn = "0022-1031",
publisher = "Academic Press Inc.",
number = "1",

}

TY - JOUR

T1 - When two become one

T2 - Temporally dynamic integration of the face and voice

AU - Freeman, Jonathan

AU - Ambady, Nalini

PY - 2011/1

Y1 - 2011/1

N2 - In everyday interactions with others, people have to deal with the sight of a face and sound of a voice at the same time. How the perceptual system brings this information together over hundreds of milliseconds to perceive others remains unclear. In 2 studies, we investigated how facial and vocal cues are integrated during real-time social categorization by recording participants' hand movements (via the streaming x, y coordinates of the computer mouse) en route to "male" and "female" responses on the screen. Participants were presented with male and female faces that were accompanied by a same-sex voice morphed to be either sex-typical (e.g., masculinized male voice) or sex-atypical (i.e., feminized male voice). Before settling into ultimate sex categorizations of the face, the simultaneous processing of a sex-atypical voice led the hand to be continuously attracted to the opposite sex-category response across construal. This is evidence that ongoing results from voice perception continuously influence face perception across processing. Thus, social categorization involves dynamic updates of gradual integration of the face and voice.

AB - In everyday interactions with others, people have to deal with the sight of a face and sound of a voice at the same time. How the perceptual system brings this information together over hundreds of milliseconds to perceive others remains unclear. In 2 studies, we investigated how facial and vocal cues are integrated during real-time social categorization by recording participants' hand movements (via the streaming x, y coordinates of the computer mouse) en route to "male" and "female" responses on the screen. Participants were presented with male and female faces that were accompanied by a same-sex voice morphed to be either sex-typical (e.g., masculinized male voice) or sex-atypical (i.e., feminized male voice). Before settling into ultimate sex categorizations of the face, the simultaneous processing of a sex-atypical voice led the hand to be continuously attracted to the opposite sex-category response across construal. This is evidence that ongoing results from voice perception continuously influence face perception across processing. Thus, social categorization involves dynamic updates of gradual integration of the face and voice.

KW - Dynamics

KW - Face perception

KW - Motor processes

KW - Person perception

KW - Social categorization

KW - Temporal

KW - Voice perception

UR - http://www.scopus.com/inward/record.url?scp=78650017199&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650017199&partnerID=8YFLogxK

U2 - 10.1016/j.jesp.2010.08.018

DO - 10.1016/j.jesp.2010.08.018

M3 - Article

VL - 47

SP - 259

EP - 263

JO - Journal of Experimental Social Psychology

JF - Journal of Experimental Social Psychology

SN - 0022-1031

IS - 1

ER -