A robust adaptive dynamic programming principle for sensorimotor control with signal-dependent noise

Research output: Contribution to journalArticle

Abstract

As human beings, people coordinate movements and interact with the environment through sensory information and motor adaptation in the daily lives. Many characteristics of these interactions can be studied using optimization-based models, which assume that the precise knowledge of both the sensorimotor system and its interactive environment is available for the central nervous system (CNS). However, both static and dynamic uncertainties occur inevitably in the daily movements. When these uncertainties are taken into consideration, the previously developed models based on optimization theory may fail to explain how the CNS can still coordinate human movements which are also robust with respect to the uncertainties. In order to address this problem, this paper presents a novel computational mechanism for sensorimotor control from a perspective of robust adaptive dynamic programming (RADP). Sharing some essential features of reinforcement learning, which was originally observed from mammals, the RADP model for sensorimotor control suggests that, instead of identifying the system dynamics of both the motor system and the environment, the CNS computes iteratively a robust optimal control policy using the real-time sensory data. An online learning algorithm is provided in this paper, with rigorous convergence and stability analysis. Then, it is applied to simulate several experiments reported from the past literature. By comparing the proposed numerical results with these experimentally observed data, the authors show that the proposed model can reproduce movement trajectories which are consistent with experimental observations. In addition, the RADP theory provides a unified framework that connects optimality and robustness properties in the sensorimotor system.

Original languageEnglish (US)
Pages (from-to)261-288
Number of pages28
JournalJournal of Systems Science and Complexity
Volume28
Issue number2
DOIs
StatePublished - 2015

Fingerprint

Dynamic programming
Neurology
Programming theory
Mammals
Reinforcement learning
Learning algorithms
Dynamical systems
Trajectories
Uncertainty
Experiments

Keywords

  • Adaptive dynamic programming
  • human motor adaptation
  • robust optimal control

ASJC Scopus subject areas

  • Information Systems
  • Computer Science (miscellaneous)

Cite this

@article{5a668a2d606c471d8a4746306a8e3155,
title = "A robust adaptive dynamic programming principle for sensorimotor control with signal-dependent noise",
abstract = "As human beings, people coordinate movements and interact with the environment through sensory information and motor adaptation in the daily lives. Many characteristics of these interactions can be studied using optimization-based models, which assume that the precise knowledge of both the sensorimotor system and its interactive environment is available for the central nervous system (CNS). However, both static and dynamic uncertainties occur inevitably in the daily movements. When these uncertainties are taken into consideration, the previously developed models based on optimization theory may fail to explain how the CNS can still coordinate human movements which are also robust with respect to the uncertainties. In order to address this problem, this paper presents a novel computational mechanism for sensorimotor control from a perspective of robust adaptive dynamic programming (RADP). Sharing some essential features of reinforcement learning, which was originally observed from mammals, the RADP model for sensorimotor control suggests that, instead of identifying the system dynamics of both the motor system and the environment, the CNS computes iteratively a robust optimal control policy using the real-time sensory data. An online learning algorithm is provided in this paper, with rigorous convergence and stability analysis. Then, it is applied to simulate several experiments reported from the past literature. By comparing the proposed numerical results with these experimentally observed data, the authors show that the proposed model can reproduce movement trajectories which are consistent with experimental observations. In addition, the RADP theory provides a unified framework that connects optimality and robustness properties in the sensorimotor system.",
keywords = "Adaptive dynamic programming, human motor adaptation, robust optimal control",
author = "Yu Jiang and Zhong-Ping Jiang",
year = "2015",
doi = "10.1007/s11424-015-3310-2",
language = "English (US)",
volume = "28",
pages = "261--288",
journal = "Journal of Systems Science and Complexity",
issn = "1009-6124",
publisher = "Springer New York",
number = "2",

}

TY - JOUR

T1 - A robust adaptive dynamic programming principle for sensorimotor control with signal-dependent noise

AU - Jiang, Yu

AU - Jiang, Zhong-Ping

PY - 2015

Y1 - 2015

N2 - As human beings, people coordinate movements and interact with the environment through sensory information and motor adaptation in the daily lives. Many characteristics of these interactions can be studied using optimization-based models, which assume that the precise knowledge of both the sensorimotor system and its interactive environment is available for the central nervous system (CNS). However, both static and dynamic uncertainties occur inevitably in the daily movements. When these uncertainties are taken into consideration, the previously developed models based on optimization theory may fail to explain how the CNS can still coordinate human movements which are also robust with respect to the uncertainties. In order to address this problem, this paper presents a novel computational mechanism for sensorimotor control from a perspective of robust adaptive dynamic programming (RADP). Sharing some essential features of reinforcement learning, which was originally observed from mammals, the RADP model for sensorimotor control suggests that, instead of identifying the system dynamics of both the motor system and the environment, the CNS computes iteratively a robust optimal control policy using the real-time sensory data. An online learning algorithm is provided in this paper, with rigorous convergence and stability analysis. Then, it is applied to simulate several experiments reported from the past literature. By comparing the proposed numerical results with these experimentally observed data, the authors show that the proposed model can reproduce movement trajectories which are consistent with experimental observations. In addition, the RADP theory provides a unified framework that connects optimality and robustness properties in the sensorimotor system.

AB - As human beings, people coordinate movements and interact with the environment through sensory information and motor adaptation in the daily lives. Many characteristics of these interactions can be studied using optimization-based models, which assume that the precise knowledge of both the sensorimotor system and its interactive environment is available for the central nervous system (CNS). However, both static and dynamic uncertainties occur inevitably in the daily movements. When these uncertainties are taken into consideration, the previously developed models based on optimization theory may fail to explain how the CNS can still coordinate human movements which are also robust with respect to the uncertainties. In order to address this problem, this paper presents a novel computational mechanism for sensorimotor control from a perspective of robust adaptive dynamic programming (RADP). Sharing some essential features of reinforcement learning, which was originally observed from mammals, the RADP model for sensorimotor control suggests that, instead of identifying the system dynamics of both the motor system and the environment, the CNS computes iteratively a robust optimal control policy using the real-time sensory data. An online learning algorithm is provided in this paper, with rigorous convergence and stability analysis. Then, it is applied to simulate several experiments reported from the past literature. By comparing the proposed numerical results with these experimentally observed data, the authors show that the proposed model can reproduce movement trajectories which are consistent with experimental observations. In addition, the RADP theory provides a unified framework that connects optimality and robustness properties in the sensorimotor system.

KW - Adaptive dynamic programming

KW - human motor adaptation

KW - robust optimal control

UR - http://www.scopus.com/inward/record.url?scp=84923784990&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923784990&partnerID=8YFLogxK

U2 - 10.1007/s11424-015-3310-2

DO - 10.1007/s11424-015-3310-2

M3 - Article

VL - 28

SP - 261

EP - 288

JO - Journal of Systems Science and Complexity

JF - Journal of Systems Science and Complexity

SN - 1009-6124

IS - 2

ER -