Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

We present preliminary results on the development of a stochastic adaptive dynamic programming theory for the data-driven, non-model-based design of robust optimal controllers for continuous-time stochastic systems. Both multiplicative noise and additive noise are considered. Two types of optimal control problems-the discounted problem and the biased problem-are investigated. Reinforcement learning and adaptive dynamic programming techniques are employed to design stochastic adaptive optimal controllers through online successive approximations of optimal solutions. Rigorous convergence proofs along with stability analysis are provided. The effectiveness of the proposed methods is validated by three illustrative practical examples arising from biological motor control and vehicle suspension control.

Original languageEnglish (US)
Title of host publicationControl of Complex Systems: Theory and Applications
PublisherElsevier Inc.
Pages211-245
Number of pages35
ISBN (Electronic)9780128054376
ISBN (Print)9780128052464
DOIs
StatePublished - Jul 23 2016

Fingerprint

Dynamic programming
Programming theory
Vehicle suspensions
Controllers
Stochastic systems
Additive noise
Reinforcement learning

Keywords

  • Dynamic programming
  • Robust adaptive dynamic programming (RADP)
  • Stochastic optimal control
  • Stochastic systems

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Bian, T., & Jiang, Z-P. (2016). Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design. In Control of Complex Systems: Theory and Applications (pp. 211-245). Elsevier Inc.. https://doi.org/10.1016/B978-0-12-805246-4.00007-0

Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design. / Bian, T.; Jiang, Zhong-Ping.

Control of Complex Systems: Theory and Applications. Elsevier Inc., 2016. p. 211-245.

Research output: Chapter in Book/Report/Conference proceedingChapter

Bian, T & Jiang, Z-P 2016, Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design. in Control of Complex Systems: Theory and Applications. Elsevier Inc., pp. 211-245. https://doi.org/10.1016/B978-0-12-805246-4.00007-0
Bian T, Jiang Z-P. Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design. In Control of Complex Systems: Theory and Applications. Elsevier Inc. 2016. p. 211-245 https://doi.org/10.1016/B978-0-12-805246-4.00007-0
Bian, T. ; Jiang, Zhong-Ping. / Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design. Control of Complex Systems: Theory and Applications. Elsevier Inc., 2016. pp. 211-245
@inbook{0b863dec90a74410a109facff87b4d36,
title = "Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design",
abstract = "We present preliminary results on the development of a stochastic adaptive dynamic programming theory for the data-driven, non-model-based design of robust optimal controllers for continuous-time stochastic systems. Both multiplicative noise and additive noise are considered. Two types of optimal control problems-the discounted problem and the biased problem-are investigated. Reinforcement learning and adaptive dynamic programming techniques are employed to design stochastic adaptive optimal controllers through online successive approximations of optimal solutions. Rigorous convergence proofs along with stability analysis are provided. The effectiveness of the proposed methods is validated by three illustrative practical examples arising from biological motor control and vehicle suspension control.",
keywords = "Dynamic programming, Robust adaptive dynamic programming (RADP), Stochastic optimal control, Stochastic systems",
author = "T. Bian and Zhong-Ping Jiang",
year = "2016",
month = "7",
day = "23",
doi = "10.1016/B978-0-12-805246-4.00007-0",
language = "English (US)",
isbn = "9780128052464",
pages = "211--245",
booktitle = "Control of Complex Systems: Theory and Applications",
publisher = "Elsevier Inc.",
address = "United States",

}

TY - CHAP

T1 - Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design

AU - Bian, T.

AU - Jiang, Zhong-Ping

PY - 2016/7/23

Y1 - 2016/7/23

N2 - We present preliminary results on the development of a stochastic adaptive dynamic programming theory for the data-driven, non-model-based design of robust optimal controllers for continuous-time stochastic systems. Both multiplicative noise and additive noise are considered. Two types of optimal control problems-the discounted problem and the biased problem-are investigated. Reinforcement learning and adaptive dynamic programming techniques are employed to design stochastic adaptive optimal controllers through online successive approximations of optimal solutions. Rigorous convergence proofs along with stability analysis are provided. The effectiveness of the proposed methods is validated by three illustrative practical examples arising from biological motor control and vehicle suspension control.

AB - We present preliminary results on the development of a stochastic adaptive dynamic programming theory for the data-driven, non-model-based design of robust optimal controllers for continuous-time stochastic systems. Both multiplicative noise and additive noise are considered. Two types of optimal control problems-the discounted problem and the biased problem-are investigated. Reinforcement learning and adaptive dynamic programming techniques are employed to design stochastic adaptive optimal controllers through online successive approximations of optimal solutions. Rigorous convergence proofs along with stability analysis are provided. The effectiveness of the proposed methods is validated by three illustrative practical examples arising from biological motor control and vehicle suspension control.

KW - Dynamic programming

KW - Robust adaptive dynamic programming (RADP)

KW - Stochastic optimal control

KW - Stochastic systems

UR - http://www.scopus.com/inward/record.url?scp=85017553163&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85017553163&partnerID=8YFLogxK

U2 - 10.1016/B978-0-12-805246-4.00007-0

DO - 10.1016/B978-0-12-805246-4.00007-0

M3 - Chapter

SN - 9780128052464

SP - 211

EP - 245

BT - Control of Complex Systems: Theory and Applications

PB - Elsevier Inc.

ER -