### Abstract

Many situations in complex systems require quantitative estimates of the lack of information in one probability distribution relative to another. In short term climate and weather prediction, examples of these issues might involve the lack of information in the historical climate record compared with an ensemble prediction, or the lack of information in a particular Gaussian ensemble prediction strategy involving the first and second moments compared with the non-Gaussian ensemble itself. The relative entropy is a natural way to quantify the predictive utility in this information, and recently a systematic computationally feasible hierarchical framework has been developed. In practical systems with many degrees of freedom, computational overhead limits ensemble predictions to relatively small sample sizes. Here the notion of predictive utility, in a relative entropy framework, is extended to small random samples by the definition of a sample utility, a measure of the unlikeliness that a random sample was produced by a given prediction strategy. The sample utility is the minimum predictability, with a statistical level of confidence, which is implied by the data. Two practical algorithms for measuring such a sample utility are developed here. The first technique is based on the statistical method of null-hypothesis testing, while the second is based upon a central limit theorem for the relative entropy of moment-based probability densities. These techniques are tested on known probability densities with parameterized bimodality and skewness, and then applied to the Lorenz '96 model, a recently developed "toy" climate model with chaotic dynamics mimicking the atmosphere. The results show a detection of non-Gaussian tendencies of prediction densities at small ensemble sizes with between 50 and 100 members, with a 95% confidence level.

Original language | English (US) |
---|---|

Pages (from-to) | 334-362 |

Number of pages | 29 |

Journal | Journal of Computational Physics |

Volume | 206 |

Issue number | 1 |

DOIs | |

State | Published - Jun 10 2005 |

### Fingerprint

### ASJC Scopus subject areas

- Computer Science Applications
- Physics and Astronomy (miscellaneous)

### Cite this

*Journal of Computational Physics*,

*206*(1), 334-362. https://doi.org/10.1016/j.jcp.2004.12.008

**Quantifying predictability through information theory : Small sample estimation in a non-Gaussian framework.** / Haven, Kyle; Majda, Andrew; Abramov, Rafail.

Research output: Contribution to journal › Article

*Journal of Computational Physics*, vol. 206, no. 1, pp. 334-362. https://doi.org/10.1016/j.jcp.2004.12.008

}

TY - JOUR

T1 - Quantifying predictability through information theory

T2 - Small sample estimation in a non-Gaussian framework

AU - Haven, Kyle

AU - Majda, Andrew

AU - Abramov, Rafail

PY - 2005/6/10

Y1 - 2005/6/10

N2 - Many situations in complex systems require quantitative estimates of the lack of information in one probability distribution relative to another. In short term climate and weather prediction, examples of these issues might involve the lack of information in the historical climate record compared with an ensemble prediction, or the lack of information in a particular Gaussian ensemble prediction strategy involving the first and second moments compared with the non-Gaussian ensemble itself. The relative entropy is a natural way to quantify the predictive utility in this information, and recently a systematic computationally feasible hierarchical framework has been developed. In practical systems with many degrees of freedom, computational overhead limits ensemble predictions to relatively small sample sizes. Here the notion of predictive utility, in a relative entropy framework, is extended to small random samples by the definition of a sample utility, a measure of the unlikeliness that a random sample was produced by a given prediction strategy. The sample utility is the minimum predictability, with a statistical level of confidence, which is implied by the data. Two practical algorithms for measuring such a sample utility are developed here. The first technique is based on the statistical method of null-hypothesis testing, while the second is based upon a central limit theorem for the relative entropy of moment-based probability densities. These techniques are tested on known probability densities with parameterized bimodality and skewness, and then applied to the Lorenz '96 model, a recently developed "toy" climate model with chaotic dynamics mimicking the atmosphere. The results show a detection of non-Gaussian tendencies of prediction densities at small ensemble sizes with between 50 and 100 members, with a 95% confidence level.

AB - Many situations in complex systems require quantitative estimates of the lack of information in one probability distribution relative to another. In short term climate and weather prediction, examples of these issues might involve the lack of information in the historical climate record compared with an ensemble prediction, or the lack of information in a particular Gaussian ensemble prediction strategy involving the first and second moments compared with the non-Gaussian ensemble itself. The relative entropy is a natural way to quantify the predictive utility in this information, and recently a systematic computationally feasible hierarchical framework has been developed. In practical systems with many degrees of freedom, computational overhead limits ensemble predictions to relatively small sample sizes. Here the notion of predictive utility, in a relative entropy framework, is extended to small random samples by the definition of a sample utility, a measure of the unlikeliness that a random sample was produced by a given prediction strategy. The sample utility is the minimum predictability, with a statistical level of confidence, which is implied by the data. Two practical algorithms for measuring such a sample utility are developed here. The first technique is based on the statistical method of null-hypothesis testing, while the second is based upon a central limit theorem for the relative entropy of moment-based probability densities. These techniques are tested on known probability densities with parameterized bimodality and skewness, and then applied to the Lorenz '96 model, a recently developed "toy" climate model with chaotic dynamics mimicking the atmosphere. The results show a detection of non-Gaussian tendencies of prediction densities at small ensemble sizes with between 50 and 100 members, with a 95% confidence level.

UR - http://www.scopus.com/inward/record.url?scp=25444532012&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=25444532012&partnerID=8YFLogxK

U2 - 10.1016/j.jcp.2004.12.008

DO - 10.1016/j.jcp.2004.12.008

M3 - Article

AN - SCOPUS:25444532012

VL - 206

SP - 334

EP - 362

JO - Journal of Computational Physics

JF - Journal of Computational Physics

SN - 0021-9991

IS - 1

ER -