### Abstract

An important emerging scientific issue is the real time filtering through observations of noisy turbulent signals for complex systems as well as the statistical accuracy of spatio-temporal discretizations for such systems. These issues are addressed here in detail for the setting with plentiful observations for a scalar field through explicit mathematical test criteria utilizing a recent theory [A.J. Majda, M.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proceedings of the National Academy of Sciences 104 (4) (2007) 1124-1129]. For plentiful observations, the number of observations equals the number of mesh points. These test criteria involve much simpler decoupled complex scalar filtering test problems with explicit formulas and elementary numerical experiments which are developed here as guidelines for filter performance. The theory includes information criteria to avoid filter divergence with large model errors, asymptotic Kalman gain, filter stability, and accurate filtering with small ensemble size as well as rigorous results delineating the role of various turbulent spectra for filtering under mesh refinement. These guidelines are also applied to discrete approximations for filtering the stochastically forced dissipative advection equation with very turbulent and noisy signals with either an equipartition of energy or -5/3 turbulent spectrum with infrequent observations as severe test problems. The theory and companion simulations demonstrate accurate statistical filtering in this context with implicit schemes with large time step with very small ensemble sizes and even with unstable explicit schemes under appropriate circumstances provided the filtering strategies are guided by the off-line theoretical criteria. The surprising failure of other strongly stable filtering strategies is also explained through these off-line criteria.

Original language | English (US) |
---|---|

Pages (from-to) | 3678-3714 |

Number of pages | 37 |

Journal | Journal of Computational Physics |

Volume | 227 |

Issue number | 7 |

DOIs | |

State | Published - Mar 20 2008 |

### Fingerprint

### Keywords

- Data assimilation
- Finite difference
- Kalman filter
- Turbulence

### ASJC Scopus subject areas

- Computer Science Applications
- Physics and Astronomy(all)

### Cite this

*Journal of Computational Physics*,

*227*(7), 3678-3714. https://doi.org/10.1016/j.jcp.2007.12.016

**Mathematical test criteria for filtering complex systems : Plentiful observations.** / Castronovo, E.; Harlim, J.; Majda, A. J.

Research output: Contribution to journal › Article

*Journal of Computational Physics*, vol. 227, no. 7, pp. 3678-3714. https://doi.org/10.1016/j.jcp.2007.12.016

}

TY - JOUR

T1 - Mathematical test criteria for filtering complex systems

T2 - Plentiful observations

AU - Castronovo, E.

AU - Harlim, J.

AU - Majda, A. J.

PY - 2008/3/20

Y1 - 2008/3/20

N2 - An important emerging scientific issue is the real time filtering through observations of noisy turbulent signals for complex systems as well as the statistical accuracy of spatio-temporal discretizations for such systems. These issues are addressed here in detail for the setting with plentiful observations for a scalar field through explicit mathematical test criteria utilizing a recent theory [A.J. Majda, M.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proceedings of the National Academy of Sciences 104 (4) (2007) 1124-1129]. For plentiful observations, the number of observations equals the number of mesh points. These test criteria involve much simpler decoupled complex scalar filtering test problems with explicit formulas and elementary numerical experiments which are developed here as guidelines for filter performance. The theory includes information criteria to avoid filter divergence with large model errors, asymptotic Kalman gain, filter stability, and accurate filtering with small ensemble size as well as rigorous results delineating the role of various turbulent spectra for filtering under mesh refinement. These guidelines are also applied to discrete approximations for filtering the stochastically forced dissipative advection equation with very turbulent and noisy signals with either an equipartition of energy or -5/3 turbulent spectrum with infrequent observations as severe test problems. The theory and companion simulations demonstrate accurate statistical filtering in this context with implicit schemes with large time step with very small ensemble sizes and even with unstable explicit schemes under appropriate circumstances provided the filtering strategies are guided by the off-line theoretical criteria. The surprising failure of other strongly stable filtering strategies is also explained through these off-line criteria.

AB - An important emerging scientific issue is the real time filtering through observations of noisy turbulent signals for complex systems as well as the statistical accuracy of spatio-temporal discretizations for such systems. These issues are addressed here in detail for the setting with plentiful observations for a scalar field through explicit mathematical test criteria utilizing a recent theory [A.J. Majda, M.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proceedings of the National Academy of Sciences 104 (4) (2007) 1124-1129]. For plentiful observations, the number of observations equals the number of mesh points. These test criteria involve much simpler decoupled complex scalar filtering test problems with explicit formulas and elementary numerical experiments which are developed here as guidelines for filter performance. The theory includes information criteria to avoid filter divergence with large model errors, asymptotic Kalman gain, filter stability, and accurate filtering with small ensemble size as well as rigorous results delineating the role of various turbulent spectra for filtering under mesh refinement. These guidelines are also applied to discrete approximations for filtering the stochastically forced dissipative advection equation with very turbulent and noisy signals with either an equipartition of energy or -5/3 turbulent spectrum with infrequent observations as severe test problems. The theory and companion simulations demonstrate accurate statistical filtering in this context with implicit schemes with large time step with very small ensemble sizes and even with unstable explicit schemes under appropriate circumstances provided the filtering strategies are guided by the off-line theoretical criteria. The surprising failure of other strongly stable filtering strategies is also explained through these off-line criteria.

KW - Data assimilation

KW - Finite difference

KW - Kalman filter

KW - Turbulence

UR - http://www.scopus.com/inward/record.url?scp=39649123724&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=39649123724&partnerID=8YFLogxK

U2 - 10.1016/j.jcp.2007.12.016

DO - 10.1016/j.jcp.2007.12.016

M3 - Article

VL - 227

SP - 3678

EP - 3714

JO - Journal of Computational Physics

JF - Journal of Computational Physics

SN - 0021-9991

IS - 7

ER -