The neural processing of hierarchical structure in music and speech at different timescales

Morwaread M. Farbood, David J. Heeger, Gary Marcus, Uri Hasson, Yulia Lerner

Research output: Contribution to journalArticle

Abstract

Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 sec) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales--by measure, phrase, and section--while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses was measured. We found that response reliability depended systematically on musical structural coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures--words/measures, sentences/musical phrases, paragraph/sections-- are processed separately.

Original languageEnglish (US)
Article number157
JournalFrontiers in Neuroscience
Volume9
Issue numberAPR
DOIs
StatePublished - 2015

Fingerprint

Music
Brain
Magnetic Resonance Imaging
Auditory Cortex
Motion Pictures
Research Design
Language

Keywords

  • fMRI
  • Hierarchical structure
  • Music
  • Processing timescales
  • Speech

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

The neural processing of hierarchical structure in music and speech at different timescales. / Farbood, Morwaread M.; Heeger, David J.; Marcus, Gary; Hasson, Uri; Lerner, Yulia.

In: Frontiers in Neuroscience, Vol. 9, No. APR, 157, 2015.

Research output: Contribution to journalArticle

@article{392cc18489d441b682025f9d55104ef0,
title = "The neural processing of hierarchical structure in music and speech at different timescales",
abstract = "Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 sec) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales--by measure, phrase, and section--while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses was measured. We found that response reliability depended systematically on musical structural coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures--words/measures, sentences/musical phrases, paragraph/sections-- are processed separately.",
keywords = "fMRI, Hierarchical structure, Music, Processing timescales, Speech",
author = "Farbood, {Morwaread M.} and Heeger, {David J.} and Gary Marcus and Uri Hasson and Yulia Lerner",
year = "2015",
doi = "10.3389/fnins.2015.00157",
language = "English (US)",
volume = "9",
journal = "Frontiers in Neuroscience",
issn = "1662-4548",
publisher = "Frontiers Research Foundation",
number = "APR",

}

TY - JOUR

T1 - The neural processing of hierarchical structure in music and speech at different timescales

AU - Farbood, Morwaread M.

AU - Heeger, David J.

AU - Marcus, Gary

AU - Hasson, Uri

AU - Lerner, Yulia

PY - 2015

Y1 - 2015

N2 - Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 sec) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales--by measure, phrase, and section--while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses was measured. We found that response reliability depended systematically on musical structural coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures--words/measures, sentences/musical phrases, paragraph/sections-- are processed separately.

AB - Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 sec) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales--by measure, phrase, and section--while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses was measured. We found that response reliability depended systematically on musical structural coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures--words/measures, sentences/musical phrases, paragraph/sections-- are processed separately.

KW - fMRI

KW - Hierarchical structure

KW - Music

KW - Processing timescales

KW - Speech

UR - http://www.scopus.com/inward/record.url?scp=84928662613&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928662613&partnerID=8YFLogxK

U2 - 10.3389/fnins.2015.00157

DO - 10.3389/fnins.2015.00157

M3 - Article

VL - 9

JO - Frontiers in Neuroscience

JF - Frontiers in Neuroscience

SN - 1662-4548

IS - APR

M1 - 157

ER -