Secrets, lies, and account recovery

Lessons from the use of personal knowledge questions at google

Joseph Bonneau, Elie Bursztein, Ilan Caron, Rob Jackson, Mike Williamson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We examine the first large real-world data set on personal knowledge question's security and memorability from their deployment at Google. Our analysis confirms that secret questions generally offer a security level that is far lower than user-chosen passwords. It turns out to be even lower than proxies such as the real distribution of surnames in the population would indicate. Surprisingly, we found that a significant cause of this insecurity is that users often don't answer truthfully. A user survey we conducted revealed that a significant fraction of users (37%) who admitted to providing fake answers did so in an attempt to make them "harder to guess" although on aggregate this behavior had the opposite effect as people "harden" their answers in a predictable way. On the usability side, we show that secret answers have surprisingly poor memorability despite the assumption that reliability motivates their continued deployment. From millions of account recovery attempts we observed a significant fraction of users (e.g 40% of our English-speaking US users) were unable to recall their answers when needed. This is lower than the success rate of alternative recovery mechanisms such as SMS reset codes (over 80%). Comparing question strength and memorability reveals that the questions that are potentially the most secure (e.g what is your first phone number) are also the ones with the worst memorability. We conclude that it appears next to impossible to find secret questions that are both secure and memorable. Secret questions continue have some use when combined with other signals, but they should not be used alone and best practice should favor more reliable alternatives.

Original languageEnglish (US)
Title of host publicationWWW 2015 - Proceedings of the 24th International Conference on World Wide Web
PublisherAssociation for Computing Machinery, Inc
Pages141-150
Number of pages10
ISBN (Electronic)9781450334693
DOIs
StatePublished - May 18 2015
Event24th International Conference on World Wide Web, WWW 2015 - Florence, Italy
Duration: May 18 2015May 22 2015

Other

Other24th International Conference on World Wide Web, WWW 2015
CountryItaly
CityFlorence
Period5/18/155/22/15

Fingerprint

Recovery

Keywords

  • Account recovery
  • Personal knowledge questions

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Bonneau, J., Bursztein, E., Caron, I., Jackson, R., & Williamson, M. (2015). Secrets, lies, and account recovery: Lessons from the use of personal knowledge questions at google. In WWW 2015 - Proceedings of the 24th International Conference on World Wide Web (pp. 141-150). Association for Computing Machinery, Inc. https://doi.org/10.1145/2736277.2741691

Secrets, lies, and account recovery : Lessons from the use of personal knowledge questions at google. / Bonneau, Joseph; Bursztein, Elie; Caron, Ilan; Jackson, Rob; Williamson, Mike.

WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Association for Computing Machinery, Inc, 2015. p. 141-150.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bonneau, J, Bursztein, E, Caron, I, Jackson, R & Williamson, M 2015, Secrets, lies, and account recovery: Lessons from the use of personal knowledge questions at google. in WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Association for Computing Machinery, Inc, pp. 141-150, 24th International Conference on World Wide Web, WWW 2015, Florence, Italy, 5/18/15. https://doi.org/10.1145/2736277.2741691
Bonneau J, Bursztein E, Caron I, Jackson R, Williamson M. Secrets, lies, and account recovery: Lessons from the use of personal knowledge questions at google. In WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Association for Computing Machinery, Inc. 2015. p. 141-150 https://doi.org/10.1145/2736277.2741691
Bonneau, Joseph ; Bursztein, Elie ; Caron, Ilan ; Jackson, Rob ; Williamson, Mike. / Secrets, lies, and account recovery : Lessons from the use of personal knowledge questions at google. WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Association for Computing Machinery, Inc, 2015. pp. 141-150
@inproceedings{47c3dec7ef774856aa376b67d3aeba29,
title = "Secrets, lies, and account recovery: Lessons from the use of personal knowledge questions at google",
abstract = "We examine the first large real-world data set on personal knowledge question's security and memorability from their deployment at Google. Our analysis confirms that secret questions generally offer a security level that is far lower than user-chosen passwords. It turns out to be even lower than proxies such as the real distribution of surnames in the population would indicate. Surprisingly, we found that a significant cause of this insecurity is that users often don't answer truthfully. A user survey we conducted revealed that a significant fraction of users (37{\%}) who admitted to providing fake answers did so in an attempt to make them {"}harder to guess{"} although on aggregate this behavior had the opposite effect as people {"}harden{"} their answers in a predictable way. On the usability side, we show that secret answers have surprisingly poor memorability despite the assumption that reliability motivates their continued deployment. From millions of account recovery attempts we observed a significant fraction of users (e.g 40{\%} of our English-speaking US users) were unable to recall their answers when needed. This is lower than the success rate of alternative recovery mechanisms such as SMS reset codes (over 80{\%}). Comparing question strength and memorability reveals that the questions that are potentially the most secure (e.g what is your first phone number) are also the ones with the worst memorability. We conclude that it appears next to impossible to find secret questions that are both secure and memorable. Secret questions continue have some use when combined with other signals, but they should not be used alone and best practice should favor more reliable alternatives.",
keywords = "Account recovery, Personal knowledge questions",
author = "Joseph Bonneau and Elie Bursztein and Ilan Caron and Rob Jackson and Mike Williamson",
year = "2015",
month = "5",
day = "18",
doi = "10.1145/2736277.2741691",
language = "English (US)",
pages = "141--150",
booktitle = "WWW 2015 - Proceedings of the 24th International Conference on World Wide Web",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Secrets, lies, and account recovery

T2 - Lessons from the use of personal knowledge questions at google

AU - Bonneau, Joseph

AU - Bursztein, Elie

AU - Caron, Ilan

AU - Jackson, Rob

AU - Williamson, Mike

PY - 2015/5/18

Y1 - 2015/5/18

N2 - We examine the first large real-world data set on personal knowledge question's security and memorability from their deployment at Google. Our analysis confirms that secret questions generally offer a security level that is far lower than user-chosen passwords. It turns out to be even lower than proxies such as the real distribution of surnames in the population would indicate. Surprisingly, we found that a significant cause of this insecurity is that users often don't answer truthfully. A user survey we conducted revealed that a significant fraction of users (37%) who admitted to providing fake answers did so in an attempt to make them "harder to guess" although on aggregate this behavior had the opposite effect as people "harden" their answers in a predictable way. On the usability side, we show that secret answers have surprisingly poor memorability despite the assumption that reliability motivates their continued deployment. From millions of account recovery attempts we observed a significant fraction of users (e.g 40% of our English-speaking US users) were unable to recall their answers when needed. This is lower than the success rate of alternative recovery mechanisms such as SMS reset codes (over 80%). Comparing question strength and memorability reveals that the questions that are potentially the most secure (e.g what is your first phone number) are also the ones with the worst memorability. We conclude that it appears next to impossible to find secret questions that are both secure and memorable. Secret questions continue have some use when combined with other signals, but they should not be used alone and best practice should favor more reliable alternatives.

AB - We examine the first large real-world data set on personal knowledge question's security and memorability from their deployment at Google. Our analysis confirms that secret questions generally offer a security level that is far lower than user-chosen passwords. It turns out to be even lower than proxies such as the real distribution of surnames in the population would indicate. Surprisingly, we found that a significant cause of this insecurity is that users often don't answer truthfully. A user survey we conducted revealed that a significant fraction of users (37%) who admitted to providing fake answers did so in an attempt to make them "harder to guess" although on aggregate this behavior had the opposite effect as people "harden" their answers in a predictable way. On the usability side, we show that secret answers have surprisingly poor memorability despite the assumption that reliability motivates their continued deployment. From millions of account recovery attempts we observed a significant fraction of users (e.g 40% of our English-speaking US users) were unable to recall their answers when needed. This is lower than the success rate of alternative recovery mechanisms such as SMS reset codes (over 80%). Comparing question strength and memorability reveals that the questions that are potentially the most secure (e.g what is your first phone number) are also the ones with the worst memorability. We conclude that it appears next to impossible to find secret questions that are both secure and memorable. Secret questions continue have some use when combined with other signals, but they should not be used alone and best practice should favor more reliable alternatives.

KW - Account recovery

KW - Personal knowledge questions

UR - http://www.scopus.com/inward/record.url?scp=84968724896&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84968724896&partnerID=8YFLogxK

U2 - 10.1145/2736277.2741691

DO - 10.1145/2736277.2741691

M3 - Conference contribution

SP - 141

EP - 150

BT - WWW 2015 - Proceedings of the 24th International Conference on World Wide Web

PB - Association for Computing Machinery, Inc

ER -