Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data

Ruizhuo Song, Frank Lewis, Qinglai Wei, Hua Guang Zhang, Zhong-Ping Jiang, Dan Levine

Research output: Contribution to journalArticle

Abstract

In industrial process control, there may be multiple performance objectives, depending on salient features of the input-output data. Aiming at this situation, this paper proposes multiple actor-critic structures to obtain the optimal control via input-output data for unknown nonlinear systems. The shunting inhibitory artificial neural network (SIANN) is used to classify the input-output data into one of several categories. Different performance measure functions may be defined for disparate categories. The approximate dynamic programming algorithm, which contains model module, critic network, and action network, is used to establish the optimal control in each category. A recurrent neural network (RNN) model is used to reconstruct the unknown system dynamics using input-output data. NNs are used to approximate the critic and action networks, respectively. It is proven that the model error and the closed unknown system are uniformly ultimately bounded. Simulation results demonstrate the performance of the proposed optimal control scheme for the unknown nonlinear system.

Original languageEnglish (US)
Article number7050362
Pages (from-to)851-865
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume26
Issue number4
DOIs
StatePublished - Apr 1 2015

Fingerprint

Nonlinear systems
Recurrent neural networks
Dynamic programming
Process control
Dynamical systems
Neural networks

Keywords

  • Actor-critic
  • approximate dynamic programming (ADP)
  • category
  • optimal control
  • shunting inhibitory artificial neural network (SIANN)

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data. / Song, Ruizhuo; Lewis, Frank; Wei, Qinglai; Zhang, Hua Guang; Jiang, Zhong-Ping; Levine, Dan.

In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, No. 4, 7050362, 01.04.2015, p. 851-865.

Research output: Contribution to journalArticle

Song, Ruizhuo ; Lewis, Frank ; Wei, Qinglai ; Zhang, Hua Guang ; Jiang, Zhong-Ping ; Levine, Dan. / Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data. In: IEEE Transactions on Neural Networks and Learning Systems. 2015 ; Vol. 26, No. 4. pp. 851-865.
@article{ae3b624afbe445039d5f3cc0162c922e,
title = "Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data",
abstract = "In industrial process control, there may be multiple performance objectives, depending on salient features of the input-output data. Aiming at this situation, this paper proposes multiple actor-critic structures to obtain the optimal control via input-output data for unknown nonlinear systems. The shunting inhibitory artificial neural network (SIANN) is used to classify the input-output data into one of several categories. Different performance measure functions may be defined for disparate categories. The approximate dynamic programming algorithm, which contains model module, critic network, and action network, is used to establish the optimal control in each category. A recurrent neural network (RNN) model is used to reconstruct the unknown system dynamics using input-output data. NNs are used to approximate the critic and action networks, respectively. It is proven that the model error and the closed unknown system are uniformly ultimately bounded. Simulation results demonstrate the performance of the proposed optimal control scheme for the unknown nonlinear system.",
keywords = "Actor-critic, approximate dynamic programming (ADP), category, optimal control, shunting inhibitory artificial neural network (SIANN)",
author = "Ruizhuo Song and Frank Lewis and Qinglai Wei and Zhang, {Hua Guang} and Zhong-Ping Jiang and Dan Levine",
year = "2015",
month = "4",
day = "1",
doi = "10.1109/TNNLS.2015.2399020",
language = "English (US)",
volume = "26",
pages = "851--865",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",
number = "4",

}

TY - JOUR

T1 - Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data

AU - Song, Ruizhuo

AU - Lewis, Frank

AU - Wei, Qinglai

AU - Zhang, Hua Guang

AU - Jiang, Zhong-Ping

AU - Levine, Dan

PY - 2015/4/1

Y1 - 2015/4/1

N2 - In industrial process control, there may be multiple performance objectives, depending on salient features of the input-output data. Aiming at this situation, this paper proposes multiple actor-critic structures to obtain the optimal control via input-output data for unknown nonlinear systems. The shunting inhibitory artificial neural network (SIANN) is used to classify the input-output data into one of several categories. Different performance measure functions may be defined for disparate categories. The approximate dynamic programming algorithm, which contains model module, critic network, and action network, is used to establish the optimal control in each category. A recurrent neural network (RNN) model is used to reconstruct the unknown system dynamics using input-output data. NNs are used to approximate the critic and action networks, respectively. It is proven that the model error and the closed unknown system are uniformly ultimately bounded. Simulation results demonstrate the performance of the proposed optimal control scheme for the unknown nonlinear system.

AB - In industrial process control, there may be multiple performance objectives, depending on salient features of the input-output data. Aiming at this situation, this paper proposes multiple actor-critic structures to obtain the optimal control via input-output data for unknown nonlinear systems. The shunting inhibitory artificial neural network (SIANN) is used to classify the input-output data into one of several categories. Different performance measure functions may be defined for disparate categories. The approximate dynamic programming algorithm, which contains model module, critic network, and action network, is used to establish the optimal control in each category. A recurrent neural network (RNN) model is used to reconstruct the unknown system dynamics using input-output data. NNs are used to approximate the critic and action networks, respectively. It is proven that the model error and the closed unknown system are uniformly ultimately bounded. Simulation results demonstrate the performance of the proposed optimal control scheme for the unknown nonlinear system.

KW - Actor-critic

KW - approximate dynamic programming (ADP)

KW - category

KW - optimal control

KW - shunting inhibitory artificial neural network (SIANN)

UR - http://www.scopus.com/inward/record.url?scp=85027929469&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027929469&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2015.2399020

DO - 10.1109/TNNLS.2015.2399020

M3 - Article

VL - 26

SP - 851

EP - 865

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 4

M1 - 7050362

ER -