Stochastic Video Generation with a Learned Prior

Emily Denton, Robert Fergus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

Original languageEnglish (US)
Title of host publication35th International Conference on Machine Learning, ICML 2018
EditorsAndreas Krause, Jennifer Dy
PublisherInternational Machine Learning Society (IMLS)
Pages1906-1919
Number of pages14
Volume3
ISBN (Electronic)9781510867963
StatePublished - Jan 1 2018
Event35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden
Duration: Jul 10 2018Jul 15 2018

Other

Other35th International Conference on Machine Learning, ICML 2018
CountrySweden
CityStockholm
Period7/10/187/15/18

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Human-Computer Interaction
  • Software

Cite this

Denton, E., & Fergus, R. (2018). Stochastic Video Generation with a Learned Prior. In A. Krause, & J. Dy (Eds.), 35th International Conference on Machine Learning, ICML 2018 (Vol. 3, pp. 1906-1919). International Machine Learning Society (IMLS).

Stochastic Video Generation with a Learned Prior. / Denton, Emily; Fergus, Robert.

35th International Conference on Machine Learning, ICML 2018. ed. / Andreas Krause; Jennifer Dy. Vol. 3 International Machine Learning Society (IMLS), 2018. p. 1906-1919.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Denton, E & Fergus, R 2018, Stochastic Video Generation with a Learned Prior. in A Krause & J Dy (eds), 35th International Conference on Machine Learning, ICML 2018. vol. 3, International Machine Learning Society (IMLS), pp. 1906-1919, 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 7/10/18.
Denton E, Fergus R. Stochastic Video Generation with a Learned Prior. In Krause A, Dy J, editors, 35th International Conference on Machine Learning, ICML 2018. Vol. 3. International Machine Learning Society (IMLS). 2018. p. 1906-1919
Denton, Emily ; Fergus, Robert. / Stochastic Video Generation with a Learned Prior. 35th International Conference on Machine Learning, ICML 2018. editor / Andreas Krause ; Jennifer Dy. Vol. 3 International Machine Learning Society (IMLS), 2018. pp. 1906-1919
@inproceedings{3ded21ab0f8f433b87e82afa68e743ff,
title = "Stochastic Video Generation with a Learned Prior",
abstract = "Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.",
author = "Emily Denton and Robert Fergus",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
volume = "3",
pages = "1906--1919",
editor = "Andreas Krause and Jennifer Dy",
booktitle = "35th International Conference on Machine Learning, ICML 2018",
publisher = "International Machine Learning Society (IMLS)",

}

TY - GEN

T1 - Stochastic Video Generation with a Learned Prior

AU - Denton, Emily

AU - Fergus, Robert

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

AB - Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

UR - http://www.scopus.com/inward/record.url?scp=85057280496&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057280496&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85057280496

VL - 3

SP - 1906

EP - 1919

BT - 35th International Conference on Machine Learning, ICML 2018

A2 - Krause, Andreas

A2 - Dy, Jennifer

PB - International Machine Learning Society (IMLS)

ER -