Reducing the Search Space for Hyperparameter Optimization Using Group Sparsity

Minsu Cho, Chinmay Hegde

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse recovery formulation, which when coupled with HyperBand (a multi-armed bandit strategy) leads to improvement over existing hyperparameter optimization methods such as Successive Halving and Random Search. Experimental results on image datasets such as CIFAR-10 confirm the benefits of our approach.

    Original languageEnglish (US)
    Title of host publication2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages3627-3631
    Number of pages5
    ISBN (Electronic)9781479981311
    DOIs
    StatePublished - May 2019
    Event44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Brighton, United Kingdom
    Duration: May 12 2019May 17 2019

    Publication series

    NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
    Volume2019-May
    ISSN (Print)1520-6149

    Conference

    Conference44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019
    CountryUnited Kingdom
    CityBrighton
    Period5/12/195/17/19

    Keywords

    • Hyperparameter optimization
    • deep learning
    • sparse recovery

    ASJC Scopus subject areas

    • Software
    • Signal Processing
    • Electrical and Electronic Engineering

    Fingerprint Dive into the research topics of 'Reducing the Search Space for Hyperparameter Optimization Using Group Sparsity'. Together they form a unique fingerprint.

  • Cite this

    Cho, M., & Hegde, C. (2019). Reducing the Search Space for Hyperparameter Optimization Using Group Sparsity. In 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings (pp. 3627-3631). [8682434] (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2019-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2019.8682434