Umbrella sampling

A powerful method to sample tails of distributions

Charles Matthews, Jonathan Weare, Andrey Kravtsov, Elise Jennings

Research output: Contribution to journalArticle

Abstract

We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerablymore robust sampling ofmultimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample the posterior accurately down to the ≈15 σ credible region in the Ωm - ΩΛ plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to ≈3 σ.

Original languageEnglish (US)
Pages (from-to)4069-4079
Number of pages11
JournalMonthly Notices of the Royal Astronomical Society
Volume480
Issue number3
DOIs
StatePublished - Jan 1 2018

Fingerprint

sampling
Markov chain
Markov chains
distribution
method
samplers
tempering
sampler
supernovae

Keywords

  • Cosmology: cosmological parameters
  • Methods: numerical

ASJC Scopus subject areas

  • Astronomy and Astrophysics
  • Space and Planetary Science

Cite this

Umbrella sampling : A powerful method to sample tails of distributions. / Matthews, Charles; Weare, Jonathan; Kravtsov, Andrey; Jennings, Elise.

In: Monthly Notices of the Royal Astronomical Society, Vol. 480, No. 3, 01.01.2018, p. 4069-4079.

Research output: Contribution to journalArticle

Matthews, Charles ; Weare, Jonathan ; Kravtsov, Andrey ; Jennings, Elise. / Umbrella sampling : A powerful method to sample tails of distributions. In: Monthly Notices of the Royal Astronomical Society. 2018 ; Vol. 480, No. 3. pp. 4069-4079.
@article{a502ecf338484ad59842ddcc214fd46d,
title = "Umbrella sampling: A powerful method to sample tails of distributions",
abstract = "We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerablymore robust sampling ofmultimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample the posterior accurately down to the ≈15 σ credible region in the Ωm - ΩΛ plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to ≈3 σ.",
keywords = "Cosmology: cosmological parameters, Methods: numerical",
author = "Charles Matthews and Jonathan Weare and Andrey Kravtsov and Elise Jennings",
year = "2018",
month = "1",
day = "1",
doi = "10.1093/mnras/sty2140",
language = "English (US)",
volume = "480",
pages = "4069--4079",
journal = "Monthly Notices of the Royal Astronomical Society",
issn = "0035-8711",
publisher = "Oxford University Press",
number = "3",

}

TY - JOUR

T1 - Umbrella sampling

T2 - A powerful method to sample tails of distributions

AU - Matthews, Charles

AU - Weare, Jonathan

AU - Kravtsov, Andrey

AU - Jennings, Elise

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerablymore robust sampling ofmultimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample the posterior accurately down to the ≈15 σ credible region in the Ωm - ΩΛ plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to ≈3 σ.

AB - We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerablymore robust sampling ofmultimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample the posterior accurately down to the ≈15 σ credible region in the Ωm - ΩΛ plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to ≈3 σ.

KW - Cosmology: cosmological parameters

KW - Methods: numerical

UR - http://www.scopus.com/inward/record.url?scp=85054734358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054734358&partnerID=8YFLogxK

U2 - 10.1093/mnras/sty2140

DO - 10.1093/mnras/sty2140

M3 - Article

VL - 480

SP - 4069

EP - 4079

JO - Monthly Notices of the Royal Astronomical Society

JF - Monthly Notices of the Royal Astronomical Society

SN - 0035-8711

IS - 3

ER -