Temporal window of integration in auditory-visual speech perception

Virginie van Wassenhove, Ken W. Grant, David Poeppel

Research output: Contribution to journalArticle

Abstract

Forty-three normal hearing participants were tested in two experiments, which focused on temporal coincidence in auditory visual (AV) speech perception. In these experiments, audio recordings of/pa/and/ba/were dubbed onto video recordings of /ba/or/ga/, respectively (ApVk, AbVg), to produce the illusory "fusion" percepts /ta/, or /da/ [McGurk, H., & McDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-747]. In Experiment 1, an identification task using McGurk pairs with asynchronies ranging from -467 ms (auditory lead) to +467 ms was conducted. Fusion responses were prevalent over temporal asynchronies from -30 ms to +170 ms and more robust for audio lags. In Experiment 2, simultaneity judgments for incongruent and congruent audiovisual tokens (AdVd, AtVt) were collected. McGurk pairs were more readily judged as asynchronous than congruent pairs. Characteristics of the temporal window over which simultaneity and fusion responses were maximal were quite similar, suggesting the existence of a 200 ms duration asymmetric bimodal temporal integration window.

Original languageEnglish (US)
Pages (from-to)598-607
Number of pages10
JournalNeuropsychologia
Volume45
Issue number3
DOIs
StatePublished - 2007

Fingerprint

Speech Perception
Visual Perception
Hearing
Auditory Perception
Video Recording
Lip

Keywords

  • Analysis-by-synthesis
  • McGurk illusion
  • Multisensory
  • Psychophysics
  • Time

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Neuropsychology and Physiological Psychology

Cite this

Temporal window of integration in auditory-visual speech perception. / van Wassenhove, Virginie; Grant, Ken W.; Poeppel, David.

In: Neuropsychologia, Vol. 45, No. 3, 2007, p. 598-607.

Research output: Contribution to journalArticle

van Wassenhove, Virginie ; Grant, Ken W. ; Poeppel, David. / Temporal window of integration in auditory-visual speech perception. In: Neuropsychologia. 2007 ; Vol. 45, No. 3. pp. 598-607.
@article{0427259f51554caea318d6cb0bc2dec5,
title = "Temporal window of integration in auditory-visual speech perception",
abstract = "Forty-three normal hearing participants were tested in two experiments, which focused on temporal coincidence in auditory visual (AV) speech perception. In these experiments, audio recordings of/pa/and/ba/were dubbed onto video recordings of /ba/or/ga/, respectively (ApVk, AbVg), to produce the illusory {"}fusion{"} percepts /ta/, or /da/ [McGurk, H., & McDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-747]. In Experiment 1, an identification task using McGurk pairs with asynchronies ranging from -467 ms (auditory lead) to +467 ms was conducted. Fusion responses were prevalent over temporal asynchronies from -30 ms to +170 ms and more robust for audio lags. In Experiment 2, simultaneity judgments for incongruent and congruent audiovisual tokens (AdVd, AtVt) were collected. McGurk pairs were more readily judged as asynchronous than congruent pairs. Characteristics of the temporal window over which simultaneity and fusion responses were maximal were quite similar, suggesting the existence of a 200 ms duration asymmetric bimodal temporal integration window.",
keywords = "Analysis-by-synthesis, McGurk illusion, Multisensory, Psychophysics, Time",
author = "{van Wassenhove}, Virginie and Grant, {Ken W.} and David Poeppel",
year = "2007",
doi = "10.1016/j.neuropsychologia.2006.01.001",
language = "English (US)",
volume = "45",
pages = "598--607",
journal = "Neuropsychologia",
issn = "0028-3932",
publisher = "Elsevier Limited",
number = "3",

}

TY - JOUR

T1 - Temporal window of integration in auditory-visual speech perception

AU - van Wassenhove, Virginie

AU - Grant, Ken W.

AU - Poeppel, David

PY - 2007

Y1 - 2007

N2 - Forty-three normal hearing participants were tested in two experiments, which focused on temporal coincidence in auditory visual (AV) speech perception. In these experiments, audio recordings of/pa/and/ba/were dubbed onto video recordings of /ba/or/ga/, respectively (ApVk, AbVg), to produce the illusory "fusion" percepts /ta/, or /da/ [McGurk, H., & McDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-747]. In Experiment 1, an identification task using McGurk pairs with asynchronies ranging from -467 ms (auditory lead) to +467 ms was conducted. Fusion responses were prevalent over temporal asynchronies from -30 ms to +170 ms and more robust for audio lags. In Experiment 2, simultaneity judgments for incongruent and congruent audiovisual tokens (AdVd, AtVt) were collected. McGurk pairs were more readily judged as asynchronous than congruent pairs. Characteristics of the temporal window over which simultaneity and fusion responses were maximal were quite similar, suggesting the existence of a 200 ms duration asymmetric bimodal temporal integration window.

AB - Forty-three normal hearing participants were tested in two experiments, which focused on temporal coincidence in auditory visual (AV) speech perception. In these experiments, audio recordings of/pa/and/ba/were dubbed onto video recordings of /ba/or/ga/, respectively (ApVk, AbVg), to produce the illusory "fusion" percepts /ta/, or /da/ [McGurk, H., & McDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-747]. In Experiment 1, an identification task using McGurk pairs with asynchronies ranging from -467 ms (auditory lead) to +467 ms was conducted. Fusion responses were prevalent over temporal asynchronies from -30 ms to +170 ms and more robust for audio lags. In Experiment 2, simultaneity judgments for incongruent and congruent audiovisual tokens (AdVd, AtVt) were collected. McGurk pairs were more readily judged as asynchronous than congruent pairs. Characteristics of the temporal window over which simultaneity and fusion responses were maximal were quite similar, suggesting the existence of a 200 ms duration asymmetric bimodal temporal integration window.

KW - Analysis-by-synthesis

KW - McGurk illusion

KW - Multisensory

KW - Psychophysics

KW - Time

UR - http://www.scopus.com/inward/record.url?scp=33845468167&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33845468167&partnerID=8YFLogxK

U2 - 10.1016/j.neuropsychologia.2006.01.001

DO - 10.1016/j.neuropsychologia.2006.01.001

M3 - Article

VL - 45

SP - 598

EP - 607

JO - Neuropsychologia

JF - Neuropsychologia

SN - 0028-3932

IS - 3

ER -