Classification with probability density estimation on sparse grids

Benjamin Peherstorfer, Fabian Franzelin, Dirk Pflüger, Hans Joachim Bungartz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel method to tackle the multi-class classification problem with sparse grids and show how the computational procedure can be split into an Offline phase (pre-processing) and a very rapid Online phase. For each class of the training data the underlying probability density function is estimated on a sparse grid. The class of a new data point is determined by the values of the density functions at this point. Our classification method can deal with more than two classes in a natural way and it provides a stochastically motivated confidence value which indicates how to rate the respond to a new point. Furthermore, the underlying density estimation method allows us to pre-compute the system matrix and store it in an appropriate format. This so-called Offline/Online splitting of the computational procedure allows an Online phase where only a few matrix vector products are necessary to learn a new, previously unseen training data set. In particular, we do not have to solve a system of linear equations anymore. We show that speed ups by a factor of several hundred are possible. A typical application for such an Offline/Online splitting is cross validation. We present the algorithm and the computational procedure for our classification method, report on the employed density estimation method on sparse grids and show by means of artificial and real world data sets that we obtain competitive results compared to the classical sparse grid classification method based on regression.

Original languageEnglish (US)
Title of host publicationSparse Grids and Applications - Munich 2012
EditorsDirk Pfluger, Jochen Garcke
PublisherSpringer-Verlag
Pages255-270
Number of pages16
ISBN (Electronic)9783319045368
StatePublished - Jan 1 2014
Event2nd Workshop on Sparse Grids and Applications, SGA 2012 - Gammarth, Tunisia
Duration: Jul 2 2012Jul 6 2012

Publication series

NameLecture Notes in Computational Science and Engineering
Volume97
ISSN (Print)1439-7358

Other

Other2nd Workshop on Sparse Grids and Applications, SGA 2012
CountryTunisia
CityGammarth
Period7/2/127/6/12

Fingerprint

Sparse Grids
Density Estimation
Probability Density
Probability density function
Linear equations
Multi-class Classification
Cross product
Matrix Product
System of Linear Equations
Cross-validation
Density Function
Classification Problems
Confidence
Preprocessing
Processing
Regression
Necessary
Class

ASJC Scopus subject areas

  • Modeling and Simulation
  • Engineering(all)
  • Discrete Mathematics and Combinatorics
  • Control and Optimization
  • Computational Mathematics

Cite this

Peherstorfer, B., Franzelin, F., Pflüger, D., & Bungartz, H. J. (2014). Classification with probability density estimation on sparse grids. In D. Pfluger, & J. Garcke (Eds.), Sparse Grids and Applications - Munich 2012 (pp. 255-270). (Lecture Notes in Computational Science and Engineering; Vol. 97). Springer-Verlag.

Classification with probability density estimation on sparse grids. / Peherstorfer, Benjamin; Franzelin, Fabian; Pflüger, Dirk; Bungartz, Hans Joachim.

Sparse Grids and Applications - Munich 2012. ed. / Dirk Pfluger; Jochen Garcke. Springer-Verlag, 2014. p. 255-270 (Lecture Notes in Computational Science and Engineering; Vol. 97).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Peherstorfer, B, Franzelin, F, Pflüger, D & Bungartz, HJ 2014, Classification with probability density estimation on sparse grids. in D Pfluger & J Garcke (eds), Sparse Grids and Applications - Munich 2012. Lecture Notes in Computational Science and Engineering, vol. 97, Springer-Verlag, pp. 255-270, 2nd Workshop on Sparse Grids and Applications, SGA 2012, Gammarth, Tunisia, 7/2/12.
Peherstorfer B, Franzelin F, Pflüger D, Bungartz HJ. Classification with probability density estimation on sparse grids. In Pfluger D, Garcke J, editors, Sparse Grids and Applications - Munich 2012. Springer-Verlag. 2014. p. 255-270. (Lecture Notes in Computational Science and Engineering).
Peherstorfer, Benjamin ; Franzelin, Fabian ; Pflüger, Dirk ; Bungartz, Hans Joachim. / Classification with probability density estimation on sparse grids. Sparse Grids and Applications - Munich 2012. editor / Dirk Pfluger ; Jochen Garcke. Springer-Verlag, 2014. pp. 255-270 (Lecture Notes in Computational Science and Engineering).
@inproceedings{e7c2bf729a7e413da5eb6e70f40c6978,
title = "Classification with probability density estimation on sparse grids",
abstract = "We present a novel method to tackle the multi-class classification problem with sparse grids and show how the computational procedure can be split into an Offline phase (pre-processing) and a very rapid Online phase. For each class of the training data the underlying probability density function is estimated on a sparse grid. The class of a new data point is determined by the values of the density functions at this point. Our classification method can deal with more than two classes in a natural way and it provides a stochastically motivated confidence value which indicates how to rate the respond to a new point. Furthermore, the underlying density estimation method allows us to pre-compute the system matrix and store it in an appropriate format. This so-called Offline/Online splitting of the computational procedure allows an Online phase where only a few matrix vector products are necessary to learn a new, previously unseen training data set. In particular, we do not have to solve a system of linear equations anymore. We show that speed ups by a factor of several hundred are possible. A typical application for such an Offline/Online splitting is cross validation. We present the algorithm and the computational procedure for our classification method, report on the employed density estimation method on sparse grids and show by means of artificial and real world data sets that we obtain competitive results compared to the classical sparse grid classification method based on regression.",
author = "Benjamin Peherstorfer and Fabian Franzelin and Dirk Pfl{\"u}ger and Bungartz, {Hans Joachim}",
year = "2014",
month = "1",
day = "1",
language = "English (US)",
series = "Lecture Notes in Computational Science and Engineering",
publisher = "Springer-Verlag",
pages = "255--270",
editor = "Dirk Pfluger and Jochen Garcke",
booktitle = "Sparse Grids and Applications - Munich 2012",

}

TY - GEN

T1 - Classification with probability density estimation on sparse grids

AU - Peherstorfer, Benjamin

AU - Franzelin, Fabian

AU - Pflüger, Dirk

AU - Bungartz, Hans Joachim

PY - 2014/1/1

Y1 - 2014/1/1

N2 - We present a novel method to tackle the multi-class classification problem with sparse grids and show how the computational procedure can be split into an Offline phase (pre-processing) and a very rapid Online phase. For each class of the training data the underlying probability density function is estimated on a sparse grid. The class of a new data point is determined by the values of the density functions at this point. Our classification method can deal with more than two classes in a natural way and it provides a stochastically motivated confidence value which indicates how to rate the respond to a new point. Furthermore, the underlying density estimation method allows us to pre-compute the system matrix and store it in an appropriate format. This so-called Offline/Online splitting of the computational procedure allows an Online phase where only a few matrix vector products are necessary to learn a new, previously unseen training data set. In particular, we do not have to solve a system of linear equations anymore. We show that speed ups by a factor of several hundred are possible. A typical application for such an Offline/Online splitting is cross validation. We present the algorithm and the computational procedure for our classification method, report on the employed density estimation method on sparse grids and show by means of artificial and real world data sets that we obtain competitive results compared to the classical sparse grid classification method based on regression.

AB - We present a novel method to tackle the multi-class classification problem with sparse grids and show how the computational procedure can be split into an Offline phase (pre-processing) and a very rapid Online phase. For each class of the training data the underlying probability density function is estimated on a sparse grid. The class of a new data point is determined by the values of the density functions at this point. Our classification method can deal with more than two classes in a natural way and it provides a stochastically motivated confidence value which indicates how to rate the respond to a new point. Furthermore, the underlying density estimation method allows us to pre-compute the system matrix and store it in an appropriate format. This so-called Offline/Online splitting of the computational procedure allows an Online phase where only a few matrix vector products are necessary to learn a new, previously unseen training data set. In particular, we do not have to solve a system of linear equations anymore. We show that speed ups by a factor of several hundred are possible. A typical application for such an Offline/Online splitting is cross validation. We present the algorithm and the computational procedure for our classification method, report on the employed density estimation method on sparse grids and show by means of artificial and real world data sets that we obtain competitive results compared to the classical sparse grid classification method based on regression.

UR - http://www.scopus.com/inward/record.url?scp=84927647167&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84927647167&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84927647167

T3 - Lecture Notes in Computational Science and Engineering

SP - 255

EP - 270

BT - Sparse Grids and Applications - Munich 2012

A2 - Pfluger, Dirk

A2 - Garcke, Jochen

PB - Springer-Verlag

ER -