Deciding where to look based on visual, auditory, and semantic information

Kyeong Jin Tark, Clayton Curtis

Research output: Contribution to journalArticle

Abstract

Neurons in the dorsal frontal and parietal cortex are thought to transform incoming visual signals into the spatial goals of saccades, a process known as target selection. Here, we used functional magnetic resonance imaging (fMRI) to test how target selection may generalize beyond visual transformations when auditory and semantic information is used for selection. We compared activity in the frontal and parietal cortex when subjects made visually, aurally, and semantically guided saccades to one of four differently colored dots. Selection was based on a visual cue (i.e., one of the dots blinked), an auditory cue (i.e., a white noise burst was emitted at one of the dots' location), or a semantic cue (i.e., the color of one of the dots was spoken). Although neural responses in frontal and parietal cortex were robust, they were non-specific with regard to the type of information used for target selection. Decoders, however, trained on the patterns of activity in the intraparietal sulcus could classify both the type of cue used for target selection and the direction of the saccade. Therefore, we find evidence that the posterior parietal cortex is involved in transforming multimodal inputs into general spatial representations that can be used to guide saccades.

Original languageEnglish (US)
Pages (from-to)26-38
Number of pages13
JournalBrain Research
Volume1525
DOIs
StatePublished - Aug 7 2013

Fingerprint

Parietal Lobe
Semantics
Saccades
Cues
Frontal Lobe
Color
Magnetic Resonance Imaging
Neurons

Keywords

  • Auditory
  • fMRI
  • Saccade
  • Semantic
  • Spatial cognition

ASJC Scopus subject areas

  • Neuroscience(all)
  • Clinical Neurology
  • Developmental Biology
  • Molecular Biology

Cite this

Deciding where to look based on visual, auditory, and semantic information. / Tark, Kyeong Jin; Curtis, Clayton.

In: Brain Research, Vol. 1525, 07.08.2013, p. 26-38.

Research output: Contribution to journalArticle

@article{9e118911e6854e7eb113bce2aa3b9cf4,
title = "Deciding where to look based on visual, auditory, and semantic information",
abstract = "Neurons in the dorsal frontal and parietal cortex are thought to transform incoming visual signals into the spatial goals of saccades, a process known as target selection. Here, we used functional magnetic resonance imaging (fMRI) to test how target selection may generalize beyond visual transformations when auditory and semantic information is used for selection. We compared activity in the frontal and parietal cortex when subjects made visually, aurally, and semantically guided saccades to one of four differently colored dots. Selection was based on a visual cue (i.e., one of the dots blinked), an auditory cue (i.e., a white noise burst was emitted at one of the dots' location), or a semantic cue (i.e., the color of one of the dots was spoken). Although neural responses in frontal and parietal cortex were robust, they were non-specific with regard to the type of information used for target selection. Decoders, however, trained on the patterns of activity in the intraparietal sulcus could classify both the type of cue used for target selection and the direction of the saccade. Therefore, we find evidence that the posterior parietal cortex is involved in transforming multimodal inputs into general spatial representations that can be used to guide saccades.",
keywords = "Auditory, fMRI, Saccade, Semantic, Spatial cognition",
author = "Tark, {Kyeong Jin} and Clayton Curtis",
year = "2013",
month = "8",
day = "7",
doi = "10.1016/j.brainres.2013.06.002",
language = "English (US)",
volume = "1525",
pages = "26--38",
journal = "Brain Research",
issn = "0006-8993",
publisher = "Elsevier",

}

TY - JOUR

T1 - Deciding where to look based on visual, auditory, and semantic information

AU - Tark, Kyeong Jin

AU - Curtis, Clayton

PY - 2013/8/7

Y1 - 2013/8/7

N2 - Neurons in the dorsal frontal and parietal cortex are thought to transform incoming visual signals into the spatial goals of saccades, a process known as target selection. Here, we used functional magnetic resonance imaging (fMRI) to test how target selection may generalize beyond visual transformations when auditory and semantic information is used for selection. We compared activity in the frontal and parietal cortex when subjects made visually, aurally, and semantically guided saccades to one of four differently colored dots. Selection was based on a visual cue (i.e., one of the dots blinked), an auditory cue (i.e., a white noise burst was emitted at one of the dots' location), or a semantic cue (i.e., the color of one of the dots was spoken). Although neural responses in frontal and parietal cortex were robust, they were non-specific with regard to the type of information used for target selection. Decoders, however, trained on the patterns of activity in the intraparietal sulcus could classify both the type of cue used for target selection and the direction of the saccade. Therefore, we find evidence that the posterior parietal cortex is involved in transforming multimodal inputs into general spatial representations that can be used to guide saccades.

AB - Neurons in the dorsal frontal and parietal cortex are thought to transform incoming visual signals into the spatial goals of saccades, a process known as target selection. Here, we used functional magnetic resonance imaging (fMRI) to test how target selection may generalize beyond visual transformations when auditory and semantic information is used for selection. We compared activity in the frontal and parietal cortex when subjects made visually, aurally, and semantically guided saccades to one of four differently colored dots. Selection was based on a visual cue (i.e., one of the dots blinked), an auditory cue (i.e., a white noise burst was emitted at one of the dots' location), or a semantic cue (i.e., the color of one of the dots was spoken). Although neural responses in frontal and parietal cortex were robust, they were non-specific with regard to the type of information used for target selection. Decoders, however, trained on the patterns of activity in the intraparietal sulcus could classify both the type of cue used for target selection and the direction of the saccade. Therefore, we find evidence that the posterior parietal cortex is involved in transforming multimodal inputs into general spatial representations that can be used to guide saccades.

KW - Auditory

KW - fMRI

KW - Saccade

KW - Semantic

KW - Spatial cognition

UR - http://www.scopus.com/inward/record.url?scp=84880328118&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880328118&partnerID=8YFLogxK

U2 - 10.1016/j.brainres.2013.06.002

DO - 10.1016/j.brainres.2013.06.002

M3 - Article

C2 - 23769862

AN - SCOPUS:84880328118

VL - 1525

SP - 26

EP - 38

JO - Brain Research

JF - Brain Research

SN - 0006-8993

ER -