Adaptive dynamic programming as a theory of sensorimotor control

Research output: Contribution to journalArticle

Abstract

Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.

Original languageEnglish (US)
Pages (from-to)459-473
Number of pages15
JournalBiological Cybernetics
Volume108
Issue number4
DOIs
StatePublished - 2014

Fingerprint

Dynamic programming
Learning
Programming theory
Systems Analysis
Uncertainty
Reinforcement learning
Neurology
Central Nervous System
Control theory
Dynamical systems
Experiments

Keywords

  • Adaptive dynamic programming
  • Endpoint stiffness
  • Motor learning
  • Optimal control

ASJC Scopus subject areas

  • Biotechnology
  • Computer Science(all)
  • Medicine(all)

Cite this

Adaptive dynamic programming as a theory of sensorimotor control. / Jiang, Yu; Jiang, Zhong-Ping.

In: Biological Cybernetics, Vol. 108, No. 4, 2014, p. 459-473.

Research output: Contribution to journalArticle

@article{071dde9ac350424699f6c541ab47b026,
title = "Adaptive dynamic programming as a theory of sensorimotor control",
abstract = "Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.",
keywords = "Adaptive dynamic programming, Endpoint stiffness, Motor learning, Optimal control",
author = "Yu Jiang and Zhong-Ping Jiang",
year = "2014",
doi = "10.1007/s00422-014-0613-7",
language = "English (US)",
volume = "108",
pages = "459--473",
journal = "Biological Cybernetics",
issn = "0340-1200",
publisher = "Springer Verlag",
number = "4",

}

TY - JOUR

T1 - Adaptive dynamic programming as a theory of sensorimotor control

AU - Jiang, Yu

AU - Jiang, Zhong-Ping

PY - 2014

Y1 - 2014

N2 - Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.

AB - Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.

KW - Adaptive dynamic programming

KW - Endpoint stiffness

KW - Motor learning

KW - Optimal control

UR - http://www.scopus.com/inward/record.url?scp=84906946687&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906946687&partnerID=8YFLogxK

U2 - 10.1007/s00422-014-0613-7

DO - 10.1007/s00422-014-0613-7

M3 - Article

C2 - 24962078

AN - SCOPUS:84906946687

VL - 108

SP - 459

EP - 473

JO - Biological Cybernetics

JF - Biological Cybernetics

SN - 0340-1200

IS - 4

ER -