Using the shapley value to analyze algorithm portfolios

Leyton Brown Kevin, Alexandre Fŕechette, Lars Kotthoff, Tomasz Michalak, Talal Rahwan, Holger H. Hoos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Algorithms for NP-complete problems often have different strengths and weaknesses, and thus algorithm portfolios often outperform individual algorithms. It is surprisingly difficult to quantify a component algorithm's contribution to such a portfolio. Reporting a component's standalone performance wrongly rewards near-clones while penalizing algorithms that have small but distinct areas of strength. Measuring a component's marginal contribution to an existing portfolio is better, but penalizes sets of strongly correlated algorithms, thereby obscuring situations in which it is essential to have at least one algorithm from such a set. This paper argues for analyzing component algorithm contributions via a measure drawn from coalitional game theory-The Shapley value-And yields insight into a research community's progress over time. We conclude with an application of the analysis we advocate to SAT competitions, yielding novel insights into the behaviour of algorithm portfolios, their components, and the state of SAT solving technology.

Original languageEnglish (US)
Title of host publication30th AAAI Conference on Artificial Intelligence, AAAI 2016
PublisherAAAI press
Pages3397-3403
Number of pages7
ISBN (Electronic)9781577357605
StatePublished - Jan 1 2016
Event30th AAAI Conference on Artificial Intelligence, AAAI 2016 - Phoenix, United States
Duration: Feb 12 2016Feb 17 2016

Other

Other30th AAAI Conference on Artificial Intelligence, AAAI 2016
CountryUnited States
CityPhoenix
Period2/12/162/17/16

    Fingerprint

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Kevin, L. B., Fŕechette, A., Kotthoff, L., Michalak, T., Rahwan, T., & Hoos, H. H. (2016). Using the shapley value to analyze algorithm portfolios. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (pp. 3397-3403). AAAI press.