Random composite forests

Giulia DeSalvo, Mehryar Mohri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We introduce a broad family of decision trees, Composite Trees, whose leaf classifiers are selected out of a hypothesis set composed of p subfamilies with different complexities. We prove new data-dependent learning guarantees for this family in the multi-class setting. These learning bounds provide a quantitative guidance for the choice of the hypotheses at each leaf. Remarkably, they depend on the Rademacher complexities of the sub-families of the predictors and the fraction of sample points correctly classified at each leaf. We further introduce random composite trees and derive learning guarantees for random composite trees which also apply to Random Forests. Using our theoretical analysis, we devise a new algorithm, RANDOMCOMPOSITEFOREST (RCF), that is based on forming an ensemble of random composite trees. We report the results of experiments demonstrating that RCF yields significant performance improvements over both Random Forests and a variant of RCF in several tasks.

Original languageEnglish (US)
Title of host publication30th AAAI Conference on Artificial Intelligence, AAAI 2016
PublisherAAAI press
Pages1540-1546
Number of pages7
ISBN (Electronic)9781577357605
StatePublished - 2016
Event30th AAAI Conference on Artificial Intelligence, AAAI 2016 - Phoenix, United States
Duration: Feb 12 2016Feb 17 2016

Other

Other30th AAAI Conference on Artificial Intelligence, AAAI 2016
CountryUnited States
CityPhoenix
Period2/12/162/17/16

Fingerprint

Composite materials
Decision trees
Classifiers
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

DeSalvo, G., & Mohri, M. (2016). Random composite forests. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (pp. 1540-1546). AAAI press.

Random composite forests. / DeSalvo, Giulia; Mohri, Mehryar.

30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, 2016. p. 1540-1546.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

DeSalvo, G & Mohri, M 2016, Random composite forests. in 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, pp. 1540-1546, 30th AAAI Conference on Artificial Intelligence, AAAI 2016, Phoenix, United States, 2/12/16.
DeSalvo G, Mohri M. Random composite forests. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press. 2016. p. 1540-1546
DeSalvo, Giulia ; Mohri, Mehryar. / Random composite forests. 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, 2016. pp. 1540-1546
@inproceedings{f03855b0c0d24566b8021f0cb334908a,
title = "Random composite forests",
abstract = "We introduce a broad family of decision trees, Composite Trees, whose leaf classifiers are selected out of a hypothesis set composed of p subfamilies with different complexities. We prove new data-dependent learning guarantees for this family in the multi-class setting. These learning bounds provide a quantitative guidance for the choice of the hypotheses at each leaf. Remarkably, they depend on the Rademacher complexities of the sub-families of the predictors and the fraction of sample points correctly classified at each leaf. We further introduce random composite trees and derive learning guarantees for random composite trees which also apply to Random Forests. Using our theoretical analysis, we devise a new algorithm, RANDOMCOMPOSITEFOREST (RCF), that is based on forming an ensemble of random composite trees. We report the results of experiments demonstrating that RCF yields significant performance improvements over both Random Forests and a variant of RCF in several tasks.",
author = "Giulia DeSalvo and Mehryar Mohri",
year = "2016",
language = "English (US)",
pages = "1540--1546",
booktitle = "30th AAAI Conference on Artificial Intelligence, AAAI 2016",
publisher = "AAAI press",

}

TY - GEN

T1 - Random composite forests

AU - DeSalvo, Giulia

AU - Mohri, Mehryar

PY - 2016

Y1 - 2016

N2 - We introduce a broad family of decision trees, Composite Trees, whose leaf classifiers are selected out of a hypothesis set composed of p subfamilies with different complexities. We prove new data-dependent learning guarantees for this family in the multi-class setting. These learning bounds provide a quantitative guidance for the choice of the hypotheses at each leaf. Remarkably, they depend on the Rademacher complexities of the sub-families of the predictors and the fraction of sample points correctly classified at each leaf. We further introduce random composite trees and derive learning guarantees for random composite trees which also apply to Random Forests. Using our theoretical analysis, we devise a new algorithm, RANDOMCOMPOSITEFOREST (RCF), that is based on forming an ensemble of random composite trees. We report the results of experiments demonstrating that RCF yields significant performance improvements over both Random Forests and a variant of RCF in several tasks.

AB - We introduce a broad family of decision trees, Composite Trees, whose leaf classifiers are selected out of a hypothesis set composed of p subfamilies with different complexities. We prove new data-dependent learning guarantees for this family in the multi-class setting. These learning bounds provide a quantitative guidance for the choice of the hypotheses at each leaf. Remarkably, they depend on the Rademacher complexities of the sub-families of the predictors and the fraction of sample points correctly classified at each leaf. We further introduce random composite trees and derive learning guarantees for random composite trees which also apply to Random Forests. Using our theoretical analysis, we devise a new algorithm, RANDOMCOMPOSITEFOREST (RCF), that is based on forming an ensemble of random composite trees. We report the results of experiments demonstrating that RCF yields significant performance improvements over both Random Forests and a variant of RCF in several tasks.

UR - http://www.scopus.com/inward/record.url?scp=85007151999&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85007151999&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85007151999

SP - 1540

EP - 1546

BT - 30th AAAI Conference on Artificial Intelligence, AAAI 2016

PB - AAAI press

ER -