Generalized Recurrent Neural Network accommodating Dynamic Causal Modeling for functional MRI analysis

Yuan Wang, Yao Wang, Yvonne W. Lui

Research output: Contribution to journalArticle

Abstract

Dynamic Causal Modeling (DCM) is an advanced biophysical model which explicitly describes the entire process from experimental stimuli to functional magnetic resonance imaging (fMRI) signals via neural activity and cerebral hemodynamics. To conduct a DCM study, one needs to represent the experimental stimuli as a compact vector-valued function of time, which is hard in complex tasks such as book reading and natural movie watching. Deep learning provides the state-of-the-art signal representation solution, encoding complex signals into compact dense vectors while preserving the essence of the original signals. There is growing interest in using Recurrent Neural Networks (RNNs), a major family of deep learning techniques, in fMRI modeling. However, the generic RNNs used in existing studies work as black boxes, making the interpretation of results in a neuroscience context difficult and obscure. In this paper, we propose a new biophysically interpretable RNN built on DCM, DCM-RNN. We generalize the vanilla RNN and show that DCM can be cast faithfully as a special form of the generalized RNN. DCM-RNN uses back propagation for parameter estimation. We believe DCM-RNN is a promising tool for neuroscience. It can fit seamlessly into classical DCM studies. We demonstrate face validity of DCM-RNN in two principal applications of DCM: causal brain architecture hypotheses testing and effective connectivity estimation. We also demonstrate construct validity of DCM-RNN in an attention-visual experiment. Moreover, DCM-RNN enables end-to-end training of DCM and representation learning deep neural networks, extending DCM studies to complex tasks.

Original languageEnglish (US)
Pages (from-to)385-402
Number of pages18
JournalNeuroImage
Volume178
DOIs
StatePublished - Sep 1 2018

Fingerprint

Magnetic Resonance Imaging
Learning
Neurosciences
Vanilla
Motion Pictures
Reproducibility of Results
Reading
Hemodynamics
Brain

Keywords

  • Causal architecture
  • Dynamic Causal Modeling
  • Effective connectivity
  • Functional magnetic resonance imaging
  • Recurrent Neural Network

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience

Cite this

Generalized Recurrent Neural Network accommodating Dynamic Causal Modeling for functional MRI analysis. / Wang, Yuan; Wang, Yao; Lui, Yvonne W.

In: NeuroImage, Vol. 178, 01.09.2018, p. 385-402.

Research output: Contribution to journalArticle

@article{c55f27b6df2541cfb5ca6c4b7d42cf3b,
title = "Generalized Recurrent Neural Network accommodating Dynamic Causal Modeling for functional MRI analysis",
abstract = "Dynamic Causal Modeling (DCM) is an advanced biophysical model which explicitly describes the entire process from experimental stimuli to functional magnetic resonance imaging (fMRI) signals via neural activity and cerebral hemodynamics. To conduct a DCM study, one needs to represent the experimental stimuli as a compact vector-valued function of time, which is hard in complex tasks such as book reading and natural movie watching. Deep learning provides the state-of-the-art signal representation solution, encoding complex signals into compact dense vectors while preserving the essence of the original signals. There is growing interest in using Recurrent Neural Networks (RNNs), a major family of deep learning techniques, in fMRI modeling. However, the generic RNNs used in existing studies work as black boxes, making the interpretation of results in a neuroscience context difficult and obscure. In this paper, we propose a new biophysically interpretable RNN built on DCM, DCM-RNN. We generalize the vanilla RNN and show that DCM can be cast faithfully as a special form of the generalized RNN. DCM-RNN uses back propagation for parameter estimation. We believe DCM-RNN is a promising tool for neuroscience. It can fit seamlessly into classical DCM studies. We demonstrate face validity of DCM-RNN in two principal applications of DCM: causal brain architecture hypotheses testing and effective connectivity estimation. We also demonstrate construct validity of DCM-RNN in an attention-visual experiment. Moreover, DCM-RNN enables end-to-end training of DCM and representation learning deep neural networks, extending DCM studies to complex tasks.",
keywords = "Causal architecture, Dynamic Causal Modeling, Effective connectivity, Functional magnetic resonance imaging, Recurrent Neural Network",
author = "Yuan Wang and Yao Wang and Lui, {Yvonne W.}",
year = "2018",
month = "9",
day = "1",
doi = "10.1016/j.neuroimage.2018.05.042",
language = "English (US)",
volume = "178",
pages = "385--402",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Generalized Recurrent Neural Network accommodating Dynamic Causal Modeling for functional MRI analysis

AU - Wang, Yuan

AU - Wang, Yao

AU - Lui, Yvonne W.

PY - 2018/9/1

Y1 - 2018/9/1

N2 - Dynamic Causal Modeling (DCM) is an advanced biophysical model which explicitly describes the entire process from experimental stimuli to functional magnetic resonance imaging (fMRI) signals via neural activity and cerebral hemodynamics. To conduct a DCM study, one needs to represent the experimental stimuli as a compact vector-valued function of time, which is hard in complex tasks such as book reading and natural movie watching. Deep learning provides the state-of-the-art signal representation solution, encoding complex signals into compact dense vectors while preserving the essence of the original signals. There is growing interest in using Recurrent Neural Networks (RNNs), a major family of deep learning techniques, in fMRI modeling. However, the generic RNNs used in existing studies work as black boxes, making the interpretation of results in a neuroscience context difficult and obscure. In this paper, we propose a new biophysically interpretable RNN built on DCM, DCM-RNN. We generalize the vanilla RNN and show that DCM can be cast faithfully as a special form of the generalized RNN. DCM-RNN uses back propagation for parameter estimation. We believe DCM-RNN is a promising tool for neuroscience. It can fit seamlessly into classical DCM studies. We demonstrate face validity of DCM-RNN in two principal applications of DCM: causal brain architecture hypotheses testing and effective connectivity estimation. We also demonstrate construct validity of DCM-RNN in an attention-visual experiment. Moreover, DCM-RNN enables end-to-end training of DCM and representation learning deep neural networks, extending DCM studies to complex tasks.

AB - Dynamic Causal Modeling (DCM) is an advanced biophysical model which explicitly describes the entire process from experimental stimuli to functional magnetic resonance imaging (fMRI) signals via neural activity and cerebral hemodynamics. To conduct a DCM study, one needs to represent the experimental stimuli as a compact vector-valued function of time, which is hard in complex tasks such as book reading and natural movie watching. Deep learning provides the state-of-the-art signal representation solution, encoding complex signals into compact dense vectors while preserving the essence of the original signals. There is growing interest in using Recurrent Neural Networks (RNNs), a major family of deep learning techniques, in fMRI modeling. However, the generic RNNs used in existing studies work as black boxes, making the interpretation of results in a neuroscience context difficult and obscure. In this paper, we propose a new biophysically interpretable RNN built on DCM, DCM-RNN. We generalize the vanilla RNN and show that DCM can be cast faithfully as a special form of the generalized RNN. DCM-RNN uses back propagation for parameter estimation. We believe DCM-RNN is a promising tool for neuroscience. It can fit seamlessly into classical DCM studies. We demonstrate face validity of DCM-RNN in two principal applications of DCM: causal brain architecture hypotheses testing and effective connectivity estimation. We also demonstrate construct validity of DCM-RNN in an attention-visual experiment. Moreover, DCM-RNN enables end-to-end training of DCM and representation learning deep neural networks, extending DCM studies to complex tasks.

KW - Causal architecture

KW - Dynamic Causal Modeling

KW - Effective connectivity

KW - Functional magnetic resonance imaging

KW - Recurrent Neural Network

UR - http://www.scopus.com/inward/record.url?scp=85047750813&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047750813&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2018.05.042

DO - 10.1016/j.neuroimage.2018.05.042

M3 - Article

C2 - 29782993

AN - SCOPUS:85047750813

VL - 178

SP - 385

EP - 402

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -