Multidimensional spectral hashing

Yair Weiss, Rob Fergus, Antonio Torralba

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the growing availability of very large image databases, there has been a surge of interest in methods based on "semantic hashing", i.e. compact binary codes of data-points so that the Hamming distance between codewords correlates with similarity. In reviewing and comparing existing methods, we show that their relative performance can change drastically depending on the definition of ground-truth neighbors. Motivated by this finding, we propose a new formulation for learning binary codes which seeks to reconstruct the affinity between datapoints, rather than their distances. We show that this criterion is intractable to solve exactly, but a spectral relaxation gives an algorithm where the bits correspond to thresholded eigenvectors of the affinity matrix, and as the number of datapoints goes to infinity these eigenvectors converge to eigenfunctions of Laplace-Beltrami operators, similar to the recently proposed Spectral Hashing (SH) method. Unlike SH whose performance may degrade as the number of bits increases, the optimal code using our formulation is guaranteed to faithfully reproduce the affinities as the number of bits increases. We show that the number of eigenfunctions needed may increase exponentially with dimension, but introduce a "kernel trick" to allow us to compute with an exponentially large number of bits but using only memory and computation that grows linearly with dimension. Experiments shows that MDSH outperforms the state-of-the art, especially in the challenging regime of small distance thresholds.

Original languageEnglish (US)
Title of host publicationComputer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings
Pages340-353
Number of pages14
Volume7576 LNCS
EditionPART 5
DOIs
StatePublished - 2012
Event12th European Conference on Computer Vision, ECCV 2012 - Florence, Italy
Duration: Oct 7 2012Oct 13 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 5
Volume7576 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other12th European Conference on Computer Vision, ECCV 2012
CountryItaly
CityFlorence
Period10/7/1210/13/12

Fingerprint

Hashing
Eigenvalues and eigenfunctions
Binary codes
Affine transformation
Binary Code
Eigenvector
Eigenfunctions
Hamming distance
Optimal Codes
Laplace-Beltrami Operator
Formulation
Surge
Hamming Distance
Image Database
Semantics
Correlate
Availability
Data storage equipment
Linearly
Infinity

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Weiss, Y., Fergus, R., & Torralba, A. (2012). Multidimensional spectral hashing. In Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings (PART 5 ed., Vol. 7576 LNCS, pp. 340-353). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7576 LNCS, No. PART 5). https://doi.org/10.1007/978-3-642-33715-4_25

Multidimensional spectral hashing. / Weiss, Yair; Fergus, Rob; Torralba, Antonio.

Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. Vol. 7576 LNCS PART 5. ed. 2012. p. 340-353 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7576 LNCS, No. PART 5).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Weiss, Y, Fergus, R & Torralba, A 2012, Multidimensional spectral hashing. in Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 5 edn, vol. 7576 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 5, vol. 7576 LNCS, pp. 340-353, 12th European Conference on Computer Vision, ECCV 2012, Florence, Italy, 10/7/12. https://doi.org/10.1007/978-3-642-33715-4_25
Weiss Y, Fergus R, Torralba A. Multidimensional spectral hashing. In Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 5 ed. Vol. 7576 LNCS. 2012. p. 340-353. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 5). https://doi.org/10.1007/978-3-642-33715-4_25
Weiss, Yair ; Fergus, Rob ; Torralba, Antonio. / Multidimensional spectral hashing. Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. Vol. 7576 LNCS PART 5. ed. 2012. pp. 340-353 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 5).
@inproceedings{35562df00fe9413bade453f495d67552,
title = "Multidimensional spectral hashing",
abstract = "With the growing availability of very large image databases, there has been a surge of interest in methods based on {"}semantic hashing{"}, i.e. compact binary codes of data-points so that the Hamming distance between codewords correlates with similarity. In reviewing and comparing existing methods, we show that their relative performance can change drastically depending on the definition of ground-truth neighbors. Motivated by this finding, we propose a new formulation for learning binary codes which seeks to reconstruct the affinity between datapoints, rather than their distances. We show that this criterion is intractable to solve exactly, but a spectral relaxation gives an algorithm where the bits correspond to thresholded eigenvectors of the affinity matrix, and as the number of datapoints goes to infinity these eigenvectors converge to eigenfunctions of Laplace-Beltrami operators, similar to the recently proposed Spectral Hashing (SH) method. Unlike SH whose performance may degrade as the number of bits increases, the optimal code using our formulation is guaranteed to faithfully reproduce the affinities as the number of bits increases. We show that the number of eigenfunctions needed may increase exponentially with dimension, but introduce a {"}kernel trick{"} to allow us to compute with an exponentially large number of bits but using only memory and computation that grows linearly with dimension. Experiments shows that MDSH outperforms the state-of-the art, especially in the challenging regime of small distance thresholds.",
author = "Yair Weiss and Rob Fergus and Antonio Torralba",
year = "2012",
doi = "10.1007/978-3-642-33715-4_25",
language = "English (US)",
isbn = "9783642337147",
volume = "7576 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
number = "PART 5",
pages = "340--353",
booktitle = "Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings",
edition = "PART 5",

}

TY - GEN

T1 - Multidimensional spectral hashing

AU - Weiss, Yair

AU - Fergus, Rob

AU - Torralba, Antonio

PY - 2012

Y1 - 2012

N2 - With the growing availability of very large image databases, there has been a surge of interest in methods based on "semantic hashing", i.e. compact binary codes of data-points so that the Hamming distance between codewords correlates with similarity. In reviewing and comparing existing methods, we show that their relative performance can change drastically depending on the definition of ground-truth neighbors. Motivated by this finding, we propose a new formulation for learning binary codes which seeks to reconstruct the affinity between datapoints, rather than their distances. We show that this criterion is intractable to solve exactly, but a spectral relaxation gives an algorithm where the bits correspond to thresholded eigenvectors of the affinity matrix, and as the number of datapoints goes to infinity these eigenvectors converge to eigenfunctions of Laplace-Beltrami operators, similar to the recently proposed Spectral Hashing (SH) method. Unlike SH whose performance may degrade as the number of bits increases, the optimal code using our formulation is guaranteed to faithfully reproduce the affinities as the number of bits increases. We show that the number of eigenfunctions needed may increase exponentially with dimension, but introduce a "kernel trick" to allow us to compute with an exponentially large number of bits but using only memory and computation that grows linearly with dimension. Experiments shows that MDSH outperforms the state-of-the art, especially in the challenging regime of small distance thresholds.

AB - With the growing availability of very large image databases, there has been a surge of interest in methods based on "semantic hashing", i.e. compact binary codes of data-points so that the Hamming distance between codewords correlates with similarity. In reviewing and comparing existing methods, we show that their relative performance can change drastically depending on the definition of ground-truth neighbors. Motivated by this finding, we propose a new formulation for learning binary codes which seeks to reconstruct the affinity between datapoints, rather than their distances. We show that this criterion is intractable to solve exactly, but a spectral relaxation gives an algorithm where the bits correspond to thresholded eigenvectors of the affinity matrix, and as the number of datapoints goes to infinity these eigenvectors converge to eigenfunctions of Laplace-Beltrami operators, similar to the recently proposed Spectral Hashing (SH) method. Unlike SH whose performance may degrade as the number of bits increases, the optimal code using our formulation is guaranteed to faithfully reproduce the affinities as the number of bits increases. We show that the number of eigenfunctions needed may increase exponentially with dimension, but introduce a "kernel trick" to allow us to compute with an exponentially large number of bits but using only memory and computation that grows linearly with dimension. Experiments shows that MDSH outperforms the state-of-the art, especially in the challenging regime of small distance thresholds.

UR - http://www.scopus.com/inward/record.url?scp=84867875948&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867875948&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-33715-4_25

DO - 10.1007/978-3-642-33715-4_25

M3 - Conference contribution

SN - 9783642337147

VL - 7576 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 340

EP - 353

BT - Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings

ER -