A Pairwise Naïve Bayes Approach to Bayesian Classification

Josephine K. Asafu-Adjei, Rebecca Betensky

Research output: Contribution to journalArticle

Abstract

Despite the relatively high accuracy of the naïve Bayes (NB) classifier, there may be several instances where it is not optimal, i.e. does not have the same classification performance as the Bayes classifier utilizing the joint distribution of the examined attributes. However, the Bayes classifier can be computationally intractable due to its required knowledge of the joint distribution. Therefore, we introduce a "pairwise naïve" Bayes (PNB) classifier that incorporates all pairwise relationships among the examined attributes, but does not require specification of the joint distribution. In this paper, we first describe the necessary and sufficient conditions under which the PNB classifier is optimal. We then discuss sufficient conditions for which the PNB classifier, and not NB, is optimal for normal attributes. Through simulation and actual studies, we evaluate the performance of our proposed classifier relative to the Bayes and NB classifiers, along with the HNB, AODE, LBR and TAN classifiers, using normal density and empirical estimation methods. Our applications show that the PNB classifier using normal density estimation yields the highest accuracy for data sets containing continuous attributes. We conclude that it offers a useful compromise between the Bayes and NB classifiers.

Original languageEnglish (US)
Article number1550023
JournalInternational Journal of Pattern Recognition and Artificial Intelligence
Volume29
Issue number7
DOIs
StatePublished - Jan 1 2015

Fingerprint

Classifiers
Specifications

Keywords

  • Bayesian classification
  • naïve Bayes classifier
  • optimal classification
  • pairwise naïve Bayes classifier
  • semi-naïve Bayes classifier

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

A Pairwise Naïve Bayes Approach to Bayesian Classification. / Asafu-Adjei, Josephine K.; Betensky, Rebecca.

In: International Journal of Pattern Recognition and Artificial Intelligence, Vol. 29, No. 7, 1550023, 01.01.2015.

Research output: Contribution to journalArticle

@article{83b8fe6ed6dd4d49833e9feb3138c22b,
title = "A Pairwise Na{\"i}ve Bayes Approach to Bayesian Classification",
abstract = "Despite the relatively high accuracy of the na{\"i}ve Bayes (NB) classifier, there may be several instances where it is not optimal, i.e. does not have the same classification performance as the Bayes classifier utilizing the joint distribution of the examined attributes. However, the Bayes classifier can be computationally intractable due to its required knowledge of the joint distribution. Therefore, we introduce a {"}pairwise na{\"i}ve{"} Bayes (PNB) classifier that incorporates all pairwise relationships among the examined attributes, but does not require specification of the joint distribution. In this paper, we first describe the necessary and sufficient conditions under which the PNB classifier is optimal. We then discuss sufficient conditions for which the PNB classifier, and not NB, is optimal for normal attributes. Through simulation and actual studies, we evaluate the performance of our proposed classifier relative to the Bayes and NB classifiers, along with the HNB, AODE, LBR and TAN classifiers, using normal density and empirical estimation methods. Our applications show that the PNB classifier using normal density estimation yields the highest accuracy for data sets containing continuous attributes. We conclude that it offers a useful compromise between the Bayes and NB classifiers.",
keywords = "Bayesian classification, na{\"i}ve Bayes classifier, optimal classification, pairwise na{\"i}ve Bayes classifier, semi-na{\"i}ve Bayes classifier",
author = "Asafu-Adjei, {Josephine K.} and Rebecca Betensky",
year = "2015",
month = "1",
day = "1",
doi = "10.1142/S0218001415500238",
language = "English (US)",
volume = "29",
journal = "International Journal of Pattern Recognition and Artificial Intelligence",
issn = "0218-0014",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "7",

}

TY - JOUR

T1 - A Pairwise Naïve Bayes Approach to Bayesian Classification

AU - Asafu-Adjei, Josephine K.

AU - Betensky, Rebecca

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Despite the relatively high accuracy of the naïve Bayes (NB) classifier, there may be several instances where it is not optimal, i.e. does not have the same classification performance as the Bayes classifier utilizing the joint distribution of the examined attributes. However, the Bayes classifier can be computationally intractable due to its required knowledge of the joint distribution. Therefore, we introduce a "pairwise naïve" Bayes (PNB) classifier that incorporates all pairwise relationships among the examined attributes, but does not require specification of the joint distribution. In this paper, we first describe the necessary and sufficient conditions under which the PNB classifier is optimal. We then discuss sufficient conditions for which the PNB classifier, and not NB, is optimal for normal attributes. Through simulation and actual studies, we evaluate the performance of our proposed classifier relative to the Bayes and NB classifiers, along with the HNB, AODE, LBR and TAN classifiers, using normal density and empirical estimation methods. Our applications show that the PNB classifier using normal density estimation yields the highest accuracy for data sets containing continuous attributes. We conclude that it offers a useful compromise between the Bayes and NB classifiers.

AB - Despite the relatively high accuracy of the naïve Bayes (NB) classifier, there may be several instances where it is not optimal, i.e. does not have the same classification performance as the Bayes classifier utilizing the joint distribution of the examined attributes. However, the Bayes classifier can be computationally intractable due to its required knowledge of the joint distribution. Therefore, we introduce a "pairwise naïve" Bayes (PNB) classifier that incorporates all pairwise relationships among the examined attributes, but does not require specification of the joint distribution. In this paper, we first describe the necessary and sufficient conditions under which the PNB classifier is optimal. We then discuss sufficient conditions for which the PNB classifier, and not NB, is optimal for normal attributes. Through simulation and actual studies, we evaluate the performance of our proposed classifier relative to the Bayes and NB classifiers, along with the HNB, AODE, LBR and TAN classifiers, using normal density and empirical estimation methods. Our applications show that the PNB classifier using normal density estimation yields the highest accuracy for data sets containing continuous attributes. We conclude that it offers a useful compromise between the Bayes and NB classifiers.

KW - Bayesian classification

KW - naïve Bayes classifier

KW - optimal classification

KW - pairwise naïve Bayes classifier

KW - semi-naïve Bayes classifier

UR - http://www.scopus.com/inward/record.url?scp=84942826208&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942826208&partnerID=8YFLogxK

U2 - 10.1142/S0218001415500238

DO - 10.1142/S0218001415500238

M3 - Article

VL - 29

JO - International Journal of Pattern Recognition and Artificial Intelligence

JF - International Journal of Pattern Recognition and Artificial Intelligence

SN - 0218-0014

IS - 7

M1 - 1550023

ER -