An adaptive Markov chain Monte Carlo approach to time series clustering of processes with regime transition behavior

Jana De Wiljes, Andrew Majda, Illia Horenko

Research output: Contribution to journalArticle

Abstract

A numerical framework for clustering of time series via a Markov chain Monte Carlo (MCMC) method is presented. It combines concepts from recently introduced variational time series analysis and regularized clustering functional minimization [I. Horenko, SIAM J. Sci. Comput., 32 (2010), pp. 62-83] with MCMC. A conceptual advantage of the presented combined framework is that it allows us to address the two main problems of the existent clustering methods, e.g., the nonconvexity and the ill-posedness of the respective functionals, in a unified way. Clustering of the time series and minimization of the regularized clustering functional are based on the generation of samples from an appropriately chosen Boltzmann distribution in the space of cluster affiliation paths using simulated annealing and the Metropolis algorithm. The presented method is applied to sets of generic ill-posed clustering problems, and the results are compared to those obtained by the standard methods. As demonstrated in numerical examples, the presented MCMC formulation of the regularized clustering problem allows us to avoid the locality of the obtained minimizers, provides good clustering results even for very ill-posed problems with overlapping clusters, and is the computationally superior method for long time series.

Original languageEnglish (US)
Pages (from-to)415-441
Number of pages27
JournalMultiscale Modeling and Simulation
Volume11
Issue number2
DOIs
StatePublished - 2013

Fingerprint

Markov chains
Markov chain
Markov Chain Monte Carlo
Markov processes
Time series
Clustering
time series
time series analysis
optimization
Time series analysis
Boltzmann distribution
simulated annealing
Simulated annealing
functionals
Monte Carlo method
Monte Carlo methods
formulations
Metropolis Algorithm
Non-convexity
Variational Analysis

Keywords

  • Clustering
  • Markov chain Monte Carlo
  • Nonstationarity
  • Regularization
  • Time series analysis

ASJC Scopus subject areas

  • Modeling and Simulation
  • Chemistry(all)
  • Computer Science Applications
  • Ecological Modeling
  • Physics and Astronomy(all)

Cite this

An adaptive Markov chain Monte Carlo approach to time series clustering of processes with regime transition behavior. / De Wiljes, Jana; Majda, Andrew; Horenko, Illia.

In: Multiscale Modeling and Simulation, Vol. 11, No. 2, 2013, p. 415-441.

Research output: Contribution to journalArticle

@article{4cba58fd2b5b4871b69e911131d3e88c,
title = "An adaptive Markov chain Monte Carlo approach to time series clustering of processes with regime transition behavior",
abstract = "A numerical framework for clustering of time series via a Markov chain Monte Carlo (MCMC) method is presented. It combines concepts from recently introduced variational time series analysis and regularized clustering functional minimization [I. Horenko, SIAM J. Sci. Comput., 32 (2010), pp. 62-83] with MCMC. A conceptual advantage of the presented combined framework is that it allows us to address the two main problems of the existent clustering methods, e.g., the nonconvexity and the ill-posedness of the respective functionals, in a unified way. Clustering of the time series and minimization of the regularized clustering functional are based on the generation of samples from an appropriately chosen Boltzmann distribution in the space of cluster affiliation paths using simulated annealing and the Metropolis algorithm. The presented method is applied to sets of generic ill-posed clustering problems, and the results are compared to those obtained by the standard methods. As demonstrated in numerical examples, the presented MCMC formulation of the regularized clustering problem allows us to avoid the locality of the obtained minimizers, provides good clustering results even for very ill-posed problems with overlapping clusters, and is the computationally superior method for long time series.",
keywords = "Clustering, Markov chain Monte Carlo, Nonstationarity, Regularization, Time series analysis",
author = "{De Wiljes}, Jana and Andrew Majda and Illia Horenko",
year = "2013",
doi = "10.1137/120881981",
language = "English (US)",
volume = "11",
pages = "415--441",
journal = "Multiscale Modeling and Simulation",
issn = "1540-3459",
publisher = "Society for Industrial and Applied Mathematics Publications",
number = "2",

}

TY - JOUR

T1 - An adaptive Markov chain Monte Carlo approach to time series clustering of processes with regime transition behavior

AU - De Wiljes, Jana

AU - Majda, Andrew

AU - Horenko, Illia

PY - 2013

Y1 - 2013

N2 - A numerical framework for clustering of time series via a Markov chain Monte Carlo (MCMC) method is presented. It combines concepts from recently introduced variational time series analysis and regularized clustering functional minimization [I. Horenko, SIAM J. Sci. Comput., 32 (2010), pp. 62-83] with MCMC. A conceptual advantage of the presented combined framework is that it allows us to address the two main problems of the existent clustering methods, e.g., the nonconvexity and the ill-posedness of the respective functionals, in a unified way. Clustering of the time series and minimization of the regularized clustering functional are based on the generation of samples from an appropriately chosen Boltzmann distribution in the space of cluster affiliation paths using simulated annealing and the Metropolis algorithm. The presented method is applied to sets of generic ill-posed clustering problems, and the results are compared to those obtained by the standard methods. As demonstrated in numerical examples, the presented MCMC formulation of the regularized clustering problem allows us to avoid the locality of the obtained minimizers, provides good clustering results even for very ill-posed problems with overlapping clusters, and is the computationally superior method for long time series.

AB - A numerical framework for clustering of time series via a Markov chain Monte Carlo (MCMC) method is presented. It combines concepts from recently introduced variational time series analysis and regularized clustering functional minimization [I. Horenko, SIAM J. Sci. Comput., 32 (2010), pp. 62-83] with MCMC. A conceptual advantage of the presented combined framework is that it allows us to address the two main problems of the existent clustering methods, e.g., the nonconvexity and the ill-posedness of the respective functionals, in a unified way. Clustering of the time series and minimization of the regularized clustering functional are based on the generation of samples from an appropriately chosen Boltzmann distribution in the space of cluster affiliation paths using simulated annealing and the Metropolis algorithm. The presented method is applied to sets of generic ill-posed clustering problems, and the results are compared to those obtained by the standard methods. As demonstrated in numerical examples, the presented MCMC formulation of the regularized clustering problem allows us to avoid the locality of the obtained minimizers, provides good clustering results even for very ill-posed problems with overlapping clusters, and is the computationally superior method for long time series.

KW - Clustering

KW - Markov chain Monte Carlo

KW - Nonstationarity

KW - Regularization

KW - Time series analysis

UR - http://www.scopus.com/inward/record.url?scp=84879698312&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84879698312&partnerID=8YFLogxK

U2 - 10.1137/120881981

DO - 10.1137/120881981

M3 - Article

AN - SCOPUS:84879698312

VL - 11

SP - 415

EP - 441

JO - Multiscale Modeling and Simulation

JF - Multiscale Modeling and Simulation

SN - 1540-3459

IS - 2

ER -