Filtering nonlinear dynamical systems with linear stochastic models

J. Harlim, A. J. Majda

Research output: Contribution to journalArticle

Abstract

An important emerging scientific issue is the real time filtering through observations of noisy signals for nonlinear dynamical systems as well as the statistical accuracy of spatio-temporal discretizations for filtering such systems. From the practical standpoint, the demand for operationally practical filtering methods escalates as the model resolution is significantly increased. For example, in numerical weather forecasting the current generation of global circulation models with resolution of 35 km has a total of billions of state variables. Numerous ensemble based Kalman filters (Evensen 2003 Ocean Dyn. 53 343-67; Bishop et al 2001 Mon. Weather Rev. 129 420-36; Anderson 2001 Mon. Weather Rev. 129 2884-903; Szunyogh et al 2005 Tellus A 57 528-45; Hunt et al 2007 Physica D 230 112-26) show promising results in addressing this issue; however, all these methods are very sensitive to model resolution, observation frequency, and the nature of the turbulent signals when a practical limited ensemble size (typically less than 100) is used. In this paper, we implement a radical filtering approach to a relatively low (40) dimensional toy model, the L-96 model (Lorenz 1996 Proc. on Predictability (ECMWF, 4-8 September 1995) pp 1-18) in various chaotic regimes in order to address the 'curse of ensemble size' for complex nonlinear systems. Practically, our approach has several desirable features such as extremely high computational efficiency, filter robustness towards variations of ensemble size (we found that the filter is reasonably stable even with a single realization) which makes it feasible for high dimensional problems, and it is independent of any tunable parameters such as the variance inflation coefficient in an ensemble Kalman filter. This radical filtering strategy decouples the problem of filtering a spatially extended nonlinear deterministic system to filtering a Fourier diagonal system of parametrized linear stochastic differential equations (Majda and Grote 2007 Proc. Natl Acad. Sci. 104 1124-9; Castronovo et al 2008 J. Comput. Phys. 227 3678-714); for the linear stochastically forced partial differential equations with constant coefficients such as in (Castronovo et al 2008 J. Comput. Phys. 227 3678-714), this Fourier diagonal decoupling is a natural approach provided that the system noise is chosen to be independent in the Fourier space; for a nonlinear problem, however, there is a strong mixing and correlations between different Fourier modes. Our strategy is to radically assume for the purposes of filtering that different Fourier modes are uncorrelated. In particular, we introduce physical model errors by replacing the nonlinearity in the original model with a suitable Ornstein-Uhlenbeck process. We show that even with this 'poor-man's' stochastic model, when the appropriate parametrization strategy is guided by mathematical offline test criteria, it is able to produce reasonably skilful filtered solutions. In the highly turbulent regime with infrequent observation time, this approach is at least as good as trusting the observations while the ensemble Kalman filter implemented in a perfect model scenario diverges. Since these Fourier diagonal linear filters have large model error compared with the nonlinear dynamics, an essential part of the study below is the interplay between this error and the mathematical criteria for a given linear filter in order to produce skilful filtered solutions through the radical strategy.

Original languageEnglish (US)
Pages (from-to)1281-1306
Number of pages26
JournalNonlinearity
Volume21
Issue number6
DOIs
StatePublished - Jun 1 2008

Fingerprint

Nonlinear dynamical systems
Nonlinear Dynamical Systems
Stochastic models
dynamical systems
Stochastic Model
Linear Model
Filtering
Ensemble
Weather
Ensemble Kalman Filter
Kalman filters
Linear Filter
Model Error
linear filters
Model
nonlinear systems
weather
Nonlinear systems
numerical weather forecasting
Filter

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics
  • Statistical and Nonlinear Physics
  • Mathematical Physics

Cite this

Filtering nonlinear dynamical systems with linear stochastic models. / Harlim, J.; Majda, A. J.

In: Nonlinearity, Vol. 21, No. 6, 01.06.2008, p. 1281-1306.

Research output: Contribution to journalArticle

@article{502b198301a64fd9822645add101df08,
title = "Filtering nonlinear dynamical systems with linear stochastic models",
abstract = "An important emerging scientific issue is the real time filtering through observations of noisy signals for nonlinear dynamical systems as well as the statistical accuracy of spatio-temporal discretizations for filtering such systems. From the practical standpoint, the demand for operationally practical filtering methods escalates as the model resolution is significantly increased. For example, in numerical weather forecasting the current generation of global circulation models with resolution of 35 km has a total of billions of state variables. Numerous ensemble based Kalman filters (Evensen 2003 Ocean Dyn. 53 343-67; Bishop et al 2001 Mon. Weather Rev. 129 420-36; Anderson 2001 Mon. Weather Rev. 129 2884-903; Szunyogh et al 2005 Tellus A 57 528-45; Hunt et al 2007 Physica D 230 112-26) show promising results in addressing this issue; however, all these methods are very sensitive to model resolution, observation frequency, and the nature of the turbulent signals when a practical limited ensemble size (typically less than 100) is used. In this paper, we implement a radical filtering approach to a relatively low (40) dimensional toy model, the L-96 model (Lorenz 1996 Proc. on Predictability (ECMWF, 4-8 September 1995) pp 1-18) in various chaotic regimes in order to address the 'curse of ensemble size' for complex nonlinear systems. Practically, our approach has several desirable features such as extremely high computational efficiency, filter robustness towards variations of ensemble size (we found that the filter is reasonably stable even with a single realization) which makes it feasible for high dimensional problems, and it is independent of any tunable parameters such as the variance inflation coefficient in an ensemble Kalman filter. This radical filtering strategy decouples the problem of filtering a spatially extended nonlinear deterministic system to filtering a Fourier diagonal system of parametrized linear stochastic differential equations (Majda and Grote 2007 Proc. Natl Acad. Sci. 104 1124-9; Castronovo et al 2008 J. Comput. Phys. 227 3678-714); for the linear stochastically forced partial differential equations with constant coefficients such as in (Castronovo et al 2008 J. Comput. Phys. 227 3678-714), this Fourier diagonal decoupling is a natural approach provided that the system noise is chosen to be independent in the Fourier space; for a nonlinear problem, however, there is a strong mixing and correlations between different Fourier modes. Our strategy is to radically assume for the purposes of filtering that different Fourier modes are uncorrelated. In particular, we introduce physical model errors by replacing the nonlinearity in the original model with a suitable Ornstein-Uhlenbeck process. We show that even with this 'poor-man's' stochastic model, when the appropriate parametrization strategy is guided by mathematical offline test criteria, it is able to produce reasonably skilful filtered solutions. In the highly turbulent regime with infrequent observation time, this approach is at least as good as trusting the observations while the ensemble Kalman filter implemented in a perfect model scenario diverges. Since these Fourier diagonal linear filters have large model error compared with the nonlinear dynamics, an essential part of the study below is the interplay between this error and the mathematical criteria for a given linear filter in order to produce skilful filtered solutions through the radical strategy.",
author = "J. Harlim and Majda, {A. J.}",
year = "2008",
month = "6",
day = "1",
doi = "10.1088/0951-7715/21/6/008",
language = "English (US)",
volume = "21",
pages = "1281--1306",
journal = "Nonlinearity",
issn = "0951-7715",
publisher = "IOP Publishing Ltd.",
number = "6",

}

TY - JOUR

T1 - Filtering nonlinear dynamical systems with linear stochastic models

AU - Harlim, J.

AU - Majda, A. J.

PY - 2008/6/1

Y1 - 2008/6/1

N2 - An important emerging scientific issue is the real time filtering through observations of noisy signals for nonlinear dynamical systems as well as the statistical accuracy of spatio-temporal discretizations for filtering such systems. From the practical standpoint, the demand for operationally practical filtering methods escalates as the model resolution is significantly increased. For example, in numerical weather forecasting the current generation of global circulation models with resolution of 35 km has a total of billions of state variables. Numerous ensemble based Kalman filters (Evensen 2003 Ocean Dyn. 53 343-67; Bishop et al 2001 Mon. Weather Rev. 129 420-36; Anderson 2001 Mon. Weather Rev. 129 2884-903; Szunyogh et al 2005 Tellus A 57 528-45; Hunt et al 2007 Physica D 230 112-26) show promising results in addressing this issue; however, all these methods are very sensitive to model resolution, observation frequency, and the nature of the turbulent signals when a practical limited ensemble size (typically less than 100) is used. In this paper, we implement a radical filtering approach to a relatively low (40) dimensional toy model, the L-96 model (Lorenz 1996 Proc. on Predictability (ECMWF, 4-8 September 1995) pp 1-18) in various chaotic regimes in order to address the 'curse of ensemble size' for complex nonlinear systems. Practically, our approach has several desirable features such as extremely high computational efficiency, filter robustness towards variations of ensemble size (we found that the filter is reasonably stable even with a single realization) which makes it feasible for high dimensional problems, and it is independent of any tunable parameters such as the variance inflation coefficient in an ensemble Kalman filter. This radical filtering strategy decouples the problem of filtering a spatially extended nonlinear deterministic system to filtering a Fourier diagonal system of parametrized linear stochastic differential equations (Majda and Grote 2007 Proc. Natl Acad. Sci. 104 1124-9; Castronovo et al 2008 J. Comput. Phys. 227 3678-714); for the linear stochastically forced partial differential equations with constant coefficients such as in (Castronovo et al 2008 J. Comput. Phys. 227 3678-714), this Fourier diagonal decoupling is a natural approach provided that the system noise is chosen to be independent in the Fourier space; for a nonlinear problem, however, there is a strong mixing and correlations between different Fourier modes. Our strategy is to radically assume for the purposes of filtering that different Fourier modes are uncorrelated. In particular, we introduce physical model errors by replacing the nonlinearity in the original model with a suitable Ornstein-Uhlenbeck process. We show that even with this 'poor-man's' stochastic model, when the appropriate parametrization strategy is guided by mathematical offline test criteria, it is able to produce reasonably skilful filtered solutions. In the highly turbulent regime with infrequent observation time, this approach is at least as good as trusting the observations while the ensemble Kalman filter implemented in a perfect model scenario diverges. Since these Fourier diagonal linear filters have large model error compared with the nonlinear dynamics, an essential part of the study below is the interplay between this error and the mathematical criteria for a given linear filter in order to produce skilful filtered solutions through the radical strategy.

AB - An important emerging scientific issue is the real time filtering through observations of noisy signals for nonlinear dynamical systems as well as the statistical accuracy of spatio-temporal discretizations for filtering such systems. From the practical standpoint, the demand for operationally practical filtering methods escalates as the model resolution is significantly increased. For example, in numerical weather forecasting the current generation of global circulation models with resolution of 35 km has a total of billions of state variables. Numerous ensemble based Kalman filters (Evensen 2003 Ocean Dyn. 53 343-67; Bishop et al 2001 Mon. Weather Rev. 129 420-36; Anderson 2001 Mon. Weather Rev. 129 2884-903; Szunyogh et al 2005 Tellus A 57 528-45; Hunt et al 2007 Physica D 230 112-26) show promising results in addressing this issue; however, all these methods are very sensitive to model resolution, observation frequency, and the nature of the turbulent signals when a practical limited ensemble size (typically less than 100) is used. In this paper, we implement a radical filtering approach to a relatively low (40) dimensional toy model, the L-96 model (Lorenz 1996 Proc. on Predictability (ECMWF, 4-8 September 1995) pp 1-18) in various chaotic regimes in order to address the 'curse of ensemble size' for complex nonlinear systems. Practically, our approach has several desirable features such as extremely high computational efficiency, filter robustness towards variations of ensemble size (we found that the filter is reasonably stable even with a single realization) which makes it feasible for high dimensional problems, and it is independent of any tunable parameters such as the variance inflation coefficient in an ensemble Kalman filter. This radical filtering strategy decouples the problem of filtering a spatially extended nonlinear deterministic system to filtering a Fourier diagonal system of parametrized linear stochastic differential equations (Majda and Grote 2007 Proc. Natl Acad. Sci. 104 1124-9; Castronovo et al 2008 J. Comput. Phys. 227 3678-714); for the linear stochastically forced partial differential equations with constant coefficients such as in (Castronovo et al 2008 J. Comput. Phys. 227 3678-714), this Fourier diagonal decoupling is a natural approach provided that the system noise is chosen to be independent in the Fourier space; for a nonlinear problem, however, there is a strong mixing and correlations between different Fourier modes. Our strategy is to radically assume for the purposes of filtering that different Fourier modes are uncorrelated. In particular, we introduce physical model errors by replacing the nonlinearity in the original model with a suitable Ornstein-Uhlenbeck process. We show that even with this 'poor-man's' stochastic model, when the appropriate parametrization strategy is guided by mathematical offline test criteria, it is able to produce reasonably skilful filtered solutions. In the highly turbulent regime with infrequent observation time, this approach is at least as good as trusting the observations while the ensemble Kalman filter implemented in a perfect model scenario diverges. Since these Fourier diagonal linear filters have large model error compared with the nonlinear dynamics, an essential part of the study below is the interplay between this error and the mathematical criteria for a given linear filter in order to produce skilful filtered solutions through the radical strategy.

UR - http://www.scopus.com/inward/record.url?scp=44949245827&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=44949245827&partnerID=8YFLogxK

U2 - 10.1088/0951-7715/21/6/008

DO - 10.1088/0951-7715/21/6/008

M3 - Article

AN - SCOPUS:44949245827

VL - 21

SP - 1281

EP - 1306

JO - Nonlinearity

JF - Nonlinearity

SN - 0951-7715

IS - 6

ER -