Model uncertainty in classical conditioning

A. C. Courville, N. D. Daw, G. J. Gordon, D. S. Touretzky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We develop a framework based on Bayesian model averaging to explain how animals cope with uncertainty about contingencies in classical conditioning experiments. Traditional accounts of conditioning fit parameters within a fixed generative model of reinforcer delivery; uncertainty over the model structure is not considered. We apply the theory to explain the puzzling relationship between second-order conditioning and conditioned inhibition, two similar conditioning regimes that nonetheless result in strongly divergent behavioral outcomes. According to the theory, second-order conditioning results when limited experience leads animals to prefer a simpler world model that produces spurious correlations; conditioned inhibition results when a more complex model is justified by additional experience.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003
PublisherNeural information processing systems foundation
ISBN (Print)0262201526, 9780262201520
StatePublished - 2004
Event17th Annual Conference on Neural Information Processing Systems, NIPS 2003 - Vancouver, BC, Canada
Duration: Dec 8 2003Dec 13 2003

Other

Other17th Annual Conference on Neural Information Processing Systems, NIPS 2003
CountryCanada
CityVancouver, BC
Period12/8/0312/13/03

Fingerprint

Animals
Model structures
Uncertainty
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Courville, A. C., Daw, N. D., Gordon, G. J., & Touretzky, D. S. (2004). Model uncertainty in classical conditioning. In Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003 Neural information processing systems foundation.

Model uncertainty in classical conditioning. / Courville, A. C.; Daw, N. D.; Gordon, G. J.; Touretzky, D. S.

Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 2004.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Courville, AC, Daw, ND, Gordon, GJ & Touretzky, DS 2004, Model uncertainty in classical conditioning. in Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 17th Annual Conference on Neural Information Processing Systems, NIPS 2003, Vancouver, BC, Canada, 12/8/03.
Courville AC, Daw ND, Gordon GJ, Touretzky DS. Model uncertainty in classical conditioning. In Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation. 2004
Courville, A. C. ; Daw, N. D. ; Gordon, G. J. ; Touretzky, D. S. / Model uncertainty in classical conditioning. Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 2004.
@inproceedings{5d7a298cc4e945f0b3a61ea9ee9d678a,
title = "Model uncertainty in classical conditioning",
abstract = "We develop a framework based on Bayesian model averaging to explain how animals cope with uncertainty about contingencies in classical conditioning experiments. Traditional accounts of conditioning fit parameters within a fixed generative model of reinforcer delivery; uncertainty over the model structure is not considered. We apply the theory to explain the puzzling relationship between second-order conditioning and conditioned inhibition, two similar conditioning regimes that nonetheless result in strongly divergent behavioral outcomes. According to the theory, second-order conditioning results when limited experience leads animals to prefer a simpler world model that produces spurious correlations; conditioned inhibition results when a more complex model is justified by additional experience.",
author = "Courville, {A. C.} and Daw, {N. D.} and Gordon, {G. J.} and Touretzky, {D. S.}",
year = "2004",
language = "English (US)",
isbn = "0262201526",
booktitle = "Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003",
publisher = "Neural information processing systems foundation",

}

TY - GEN

T1 - Model uncertainty in classical conditioning

AU - Courville, A. C.

AU - Daw, N. D.

AU - Gordon, G. J.

AU - Touretzky, D. S.

PY - 2004

Y1 - 2004

N2 - We develop a framework based on Bayesian model averaging to explain how animals cope with uncertainty about contingencies in classical conditioning experiments. Traditional accounts of conditioning fit parameters within a fixed generative model of reinforcer delivery; uncertainty over the model structure is not considered. We apply the theory to explain the puzzling relationship between second-order conditioning and conditioned inhibition, two similar conditioning regimes that nonetheless result in strongly divergent behavioral outcomes. According to the theory, second-order conditioning results when limited experience leads animals to prefer a simpler world model that produces spurious correlations; conditioned inhibition results when a more complex model is justified by additional experience.

AB - We develop a framework based on Bayesian model averaging to explain how animals cope with uncertainty about contingencies in classical conditioning experiments. Traditional accounts of conditioning fit parameters within a fixed generative model of reinforcer delivery; uncertainty over the model structure is not considered. We apply the theory to explain the puzzling relationship between second-order conditioning and conditioned inhibition, two similar conditioning regimes that nonetheless result in strongly divergent behavioral outcomes. According to the theory, second-order conditioning results when limited experience leads animals to prefer a simpler world model that produces spurious correlations; conditioned inhibition results when a more complex model is justified by additional experience.

UR - http://www.scopus.com/inward/record.url?scp=84898993037&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898993037&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0262201526

SN - 9780262201520

BT - Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003

PB - Neural information processing systems foundation

ER -