Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems

Research output: Contribution to journalArticle

Abstract

Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model. The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing K-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.

Original languageEnglish (US)
Pages (from-to)1735-1752
Number of pages18
JournalPhysica D: Nonlinear Phenomena
Volume241
Issue number20
DOIs
StatePublished - Oct 15 2012

Fingerprint

information theory
nonlinear systems
statistics
smoothing
forecasting
dynamical systems
partitions
education
molecular dynamics
predictions
approximation

Keywords

  • Autoregressive models
  • Clustering algorithms
  • Information theory
  • Model error
  • Predictability
  • Stochastic models

ASJC Scopus subject areas

  • Condensed Matter Physics
  • Statistical and Nonlinear Physics

Cite this

Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems. / Giannakis, Dimitrios; Majda, Andrew J.; Horenko, Illia.

In: Physica D: Nonlinear Phenomena, Vol. 241, No. 20, 15.10.2012, p. 1735-1752.

Research output: Contribution to journalArticle

@article{96dbfe835eb84ac7a962098873b7cdde,
title = "Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems",
abstract = "Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model. The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing K-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.",
keywords = "Autoregressive models, Clustering algorithms, Information theory, Model error, Predictability, Stochastic models",
author = "Dimitrios Giannakis and Majda, {Andrew J.} and Illia Horenko",
year = "2012",
month = "10",
day = "15",
doi = "10.1016/j.physd.2012.07.005",
language = "English (US)",
volume = "241",
pages = "1735--1752",
journal = "Physica D: Nonlinear Phenomena",
issn = "0167-2789",
publisher = "Elsevier",
number = "20",

}

TY - JOUR

T1 - Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems

AU - Giannakis, Dimitrios

AU - Majda, Andrew J.

AU - Horenko, Illia

PY - 2012/10/15

Y1 - 2012/10/15

N2 - Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model. The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing K-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.

AB - Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model. The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing K-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.

KW - Autoregressive models

KW - Clustering algorithms

KW - Information theory

KW - Model error

KW - Predictability

KW - Stochastic models

UR - http://www.scopus.com/inward/record.url?scp=84865980981&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865980981&partnerID=8YFLogxK

U2 - 10.1016/j.physd.2012.07.005

DO - 10.1016/j.physd.2012.07.005

M3 - Article

VL - 241

SP - 1735

EP - 1752

JO - Physica D: Nonlinear Phenomena

JF - Physica D: Nonlinear Phenomena

SN - 0167-2789

IS - 20

ER -