An alternative ranking problem for search engines

Corinna Cortes, Mehryar Mohri, Ashish Rastogi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper examines in detail an alternative ranking problem for search engines, movie recommendation, and other similar ranking systems motivated by the requirement to not just accurately predict pairwise ordering but also preserve the magnitude of the preferences or the difference between ratings. We describe and analyze several cost functions for this learning problem and give stability bounds for their generalization error, extending previously known stability results to non-bipartite ranking and magnitude of preference- preserving algorithms. We present algorithms optimizing these cost functions, and, in one instance, detail both a batch and an on-line version. For this algorithm, we also show how the leave-one-out error can be computed and approximated efficiently, which can be used to determine the optimal values of the trade-off parameter in the cost function. We report the results of experiments comparing these algorithms on several datasets and contrast them with those obtained using an AUC-maximization algorithm. We also compare training times and performance results for the on-line and batch versions, demonstrating that our on-line algorithm scales to relatively large datasets with no significant loss in accuracy.

Original languageEnglish (US)
Title of host publicationExperimental Algorithms - 6th International Workshop, WEA 2007, Proceedings
Pages1-22
Number of pages22
Volume4525 LNCS
StatePublished - 2007
Event6th International Workshop on Experimental Algorithms, WEA 2007 - Rome, Italy
Duration: Jun 6 2007Jun 8 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4525 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other6th International Workshop on Experimental Algorithms, WEA 2007
CountryItaly
CityRome
Period6/6/076/8/07

Fingerprint

Search Engine
Search engines
Ranking
Alternatives
Cost functions
Cost Function
Costs and Cost Analysis
Batch
Generalization Error
Motion Pictures
Large Data Sets
Area Under Curve
Pairwise
Recommendations
Trade-offs
Learning
Predict
Requirements
Experiment

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Cortes, C., Mohri, M., & Rastogi, A. (2007). An alternative ranking problem for search engines. In Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings (Vol. 4525 LNCS, pp. 1-22). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4525 LNCS).

An alternative ranking problem for search engines. / Cortes, Corinna; Mohri, Mehryar; Rastogi, Ashish.

Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings. Vol. 4525 LNCS 2007. p. 1-22 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4525 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Cortes, C, Mohri, M & Rastogi, A 2007, An alternative ranking problem for search engines. in Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings. vol. 4525 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4525 LNCS, pp. 1-22, 6th International Workshop on Experimental Algorithms, WEA 2007, Rome, Italy, 6/6/07.
Cortes C, Mohri M, Rastogi A. An alternative ranking problem for search engines. In Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings. Vol. 4525 LNCS. 2007. p. 1-22. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Cortes, Corinna ; Mohri, Mehryar ; Rastogi, Ashish. / An alternative ranking problem for search engines. Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings. Vol. 4525 LNCS 2007. pp. 1-22 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{b898059a34b547fca741ac430414e68d,
title = "An alternative ranking problem for search engines",
abstract = "This paper examines in detail an alternative ranking problem for search engines, movie recommendation, and other similar ranking systems motivated by the requirement to not just accurately predict pairwise ordering but also preserve the magnitude of the preferences or the difference between ratings. We describe and analyze several cost functions for this learning problem and give stability bounds for their generalization error, extending previously known stability results to non-bipartite ranking and magnitude of preference- preserving algorithms. We present algorithms optimizing these cost functions, and, in one instance, detail both a batch and an on-line version. For this algorithm, we also show how the leave-one-out error can be computed and approximated efficiently, which can be used to determine the optimal values of the trade-off parameter in the cost function. We report the results of experiments comparing these algorithms on several datasets and contrast them with those obtained using an AUC-maximization algorithm. We also compare training times and performance results for the on-line and batch versions, demonstrating that our on-line algorithm scales to relatively large datasets with no significant loss in accuracy.",
author = "Corinna Cortes and Mehryar Mohri and Ashish Rastogi",
year = "2007",
language = "English (US)",
isbn = "3540728449",
volume = "4525 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "1--22",
booktitle = "Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings",

}

TY - GEN

T1 - An alternative ranking problem for search engines

AU - Cortes, Corinna

AU - Mohri, Mehryar

AU - Rastogi, Ashish

PY - 2007

Y1 - 2007

N2 - This paper examines in detail an alternative ranking problem for search engines, movie recommendation, and other similar ranking systems motivated by the requirement to not just accurately predict pairwise ordering but also preserve the magnitude of the preferences or the difference between ratings. We describe and analyze several cost functions for this learning problem and give stability bounds for their generalization error, extending previously known stability results to non-bipartite ranking and magnitude of preference- preserving algorithms. We present algorithms optimizing these cost functions, and, in one instance, detail both a batch and an on-line version. For this algorithm, we also show how the leave-one-out error can be computed and approximated efficiently, which can be used to determine the optimal values of the trade-off parameter in the cost function. We report the results of experiments comparing these algorithms on several datasets and contrast them with those obtained using an AUC-maximization algorithm. We also compare training times and performance results for the on-line and batch versions, demonstrating that our on-line algorithm scales to relatively large datasets with no significant loss in accuracy.

AB - This paper examines in detail an alternative ranking problem for search engines, movie recommendation, and other similar ranking systems motivated by the requirement to not just accurately predict pairwise ordering but also preserve the magnitude of the preferences or the difference between ratings. We describe and analyze several cost functions for this learning problem and give stability bounds for their generalization error, extending previously known stability results to non-bipartite ranking and magnitude of preference- preserving algorithms. We present algorithms optimizing these cost functions, and, in one instance, detail both a batch and an on-line version. For this algorithm, we also show how the leave-one-out error can be computed and approximated efficiently, which can be used to determine the optimal values of the trade-off parameter in the cost function. We report the results of experiments comparing these algorithms on several datasets and contrast them with those obtained using an AUC-maximization algorithm. We also compare training times and performance results for the on-line and batch versions, demonstrating that our on-line algorithm scales to relatively large datasets with no significant loss in accuracy.

UR - http://www.scopus.com/inward/record.url?scp=37149011781&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=37149011781&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:37149011781

SN - 3540728449

SN - 9783540728443

VL - 4525 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1

EP - 22

BT - Experimental Algorithms - 6th International Workshop, WEA 2007, Proceedings

ER -