Very Long Term Field of View Prediction for 360-Degree Video Streaming

Chenge Li, Weixi Zhang, Yong Liu, Yao Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.

Original languageEnglish (US)
Title of host publicationProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages297-302
Number of pages6
ISBN (Electronic)9781728111988
DOIs
StatePublished - Apr 22 2019
Event2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States
Duration: Mar 28 2019Mar 30 2019

Publication series

NameProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Conference

Conference2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
CountryUnited States
CitySan Jose
Period3/28/193/30/19

Fingerprint

Video streaming
Trajectories
Bandwidth
Video on demand
Augmented reality
Watches
Freezing
Virtual reality

Keywords

  • 360-degree video streaming
  • field of view
  • time series prediction
  • virtual reality

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Signal Processing
  • Media Technology

Cite this

Li, C., Zhang, W., Liu, Y., & Wang, Y. (2019). Very Long Term Field of View Prediction for 360-Degree Video Streaming. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 (pp. 297-302). [8695336] (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MIPR.2019.00060

Very Long Term Field of View Prediction for 360-Degree Video Streaming. / Li, Chenge; Zhang, Weixi; Liu, Yong; Wang, Yao.

Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 297-302 8695336 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, C, Zhang, W, Liu, Y & Wang, Y 2019, Very Long Term Field of View Prediction for 360-Degree Video Streaming. in Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019., 8695336, Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, Institute of Electrical and Electronics Engineers Inc., pp. 297-302, 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, San Jose, United States, 3/28/19. https://doi.org/10.1109/MIPR.2019.00060
Li C, Zhang W, Liu Y, Wang Y. Very Long Term Field of View Prediction for 360-Degree Video Streaming. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 297-302. 8695336. (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). https://doi.org/10.1109/MIPR.2019.00060
Li, Chenge ; Zhang, Weixi ; Liu, Yong ; Wang, Yao. / Very Long Term Field of View Prediction for 360-Degree Video Streaming. Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 297-302 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).
@inproceedings{3358dcac6ae0460fa64c5f6ba59b8045,
title = "Very Long Term Field of View Prediction for 360-Degree Video Streaming",
abstract = "360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.",
keywords = "360-degree video streaming, field of view, time series prediction, virtual reality",
author = "Chenge Li and Weixi Zhang and Yong Liu and Yao Wang",
year = "2019",
month = "4",
day = "22",
doi = "10.1109/MIPR.2019.00060",
language = "English (US)",
series = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "297--302",
booktitle = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",

}

TY - GEN

T1 - Very Long Term Field of View Prediction for 360-Degree Video Streaming

AU - Li, Chenge

AU - Zhang, Weixi

AU - Liu, Yong

AU - Wang, Yao

PY - 2019/4/22

Y1 - 2019/4/22

N2 - 360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.

AB - 360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.

KW - 360-degree video streaming

KW - field of view

KW - time series prediction

KW - virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85065627631&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065627631&partnerID=8YFLogxK

U2 - 10.1109/MIPR.2019.00060

DO - 10.1109/MIPR.2019.00060

M3 - Conference contribution

T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

SP - 297

EP - 302

BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -