Perceptual Quality Maximization for Video Calls with Packet Losses by Optimizing FEC, Frame Rate and Quantization

Eymen Kurdoglu, Yong Liu, Yao Wang

Research output: Contribution to journalArticle

Abstract

We consider video calls affected by bursty packet losses, where FEC is applied on a per-frame basis due to the extremely tight delay constraints. In this scenario, a high frame rate at low bitrates leads to large quantization step-sizes (QS), small frames and suboptimal FEC, while a low frame rate at high video bitrates reduces the perceptual quality. At the receiver, damaged frames and others predicted from them are typically discarded, lowering the decoded frame rate. To mitigate frame losses and the following freezing events, temporal layering via hierarchical-P coding structure (hierP) can be used at the cost of lower coding efficiency compared to the traditional IPPP coding structure. In this paper, we study the received video quality maximization for both hierP and IPPP by jointly optimizing the encoding frame rate, QS and the FEC redundancy rates, under the sending bitrate constraint. Building upon Q-STAR, a perceptual video quality model that depends on QS and the decoded frame rate, along with R-STAR, a video bitrate model that depends on QS and the encoding frame rate, we cast the problem as a combinatorial optimization problem. We then solve for the encoding frame rate and the video bitrate using exhaustive search and hill-climbing, as well as a greedy FEC packet distribution algorithm to determine the FEC redundancy rate for each frame. We show that, for independent and identically distributed random losses, (a) the FEC bitrate ratio is an affine function of the packet loss rate, (b) low encoding frame rates are preferred at wider sending bitrate ranges with more severe packet losses, (c) layers are protected more evenly at higher bitrates, and (d) IPPP, while achieving higher Q-STAR scores, is prone to abrupt freezing that is not considered by the Q-STAR model. For bursty losses, we show that (a) layer redundancies are much higher, rising with the mean burst length and reaching up to 80%, (b) hierP achieves higher Q-STAR scores than IPPP in case of longer bursts, and (c) the mean and the variance of decoded frame distances are significantly smaller with hierP.

Original languageEnglish (US)
JournalIEEE Transactions on Multimedia
DOIs
StateAccepted/In press - Dec 7 2017

Fingerprint

Packet loss
Redundancy
Freezing
Combinatorial optimization

Keywords

  • Bit rate
  • Delays
  • Encoding
  • Forward error correction
  • Packet loss
  • Streaming media

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

@article{da7c7a95ce8e4d82bae3977bdd38d55b,
title = "Perceptual Quality Maximization for Video Calls with Packet Losses by Optimizing FEC, Frame Rate and Quantization",
abstract = "We consider video calls affected by bursty packet losses, where FEC is applied on a per-frame basis due to the extremely tight delay constraints. In this scenario, a high frame rate at low bitrates leads to large quantization step-sizes (QS), small frames and suboptimal FEC, while a low frame rate at high video bitrates reduces the perceptual quality. At the receiver, damaged frames and others predicted from them are typically discarded, lowering the decoded frame rate. To mitigate frame losses and the following freezing events, temporal layering via hierarchical-P coding structure (hierP) can be used at the cost of lower coding efficiency compared to the traditional IPPP coding structure. In this paper, we study the received video quality maximization for both hierP and IPPP by jointly optimizing the encoding frame rate, QS and the FEC redundancy rates, under the sending bitrate constraint. Building upon Q-STAR, a perceptual video quality model that depends on QS and the decoded frame rate, along with R-STAR, a video bitrate model that depends on QS and the encoding frame rate, we cast the problem as a combinatorial optimization problem. We then solve for the encoding frame rate and the video bitrate using exhaustive search and hill-climbing, as well as a greedy FEC packet distribution algorithm to determine the FEC redundancy rate for each frame. We show that, for independent and identically distributed random losses, (a) the FEC bitrate ratio is an affine function of the packet loss rate, (b) low encoding frame rates are preferred at wider sending bitrate ranges with more severe packet losses, (c) layers are protected more evenly at higher bitrates, and (d) IPPP, while achieving higher Q-STAR scores, is prone to abrupt freezing that is not considered by the Q-STAR model. For bursty losses, we show that (a) layer redundancies are much higher, rising with the mean burst length and reaching up to 80{\%}, (b) hierP achieves higher Q-STAR scores than IPPP in case of longer bursts, and (c) the mean and the variance of decoded frame distances are significantly smaller with hierP.",
keywords = "Bit rate, Delays, Encoding, Forward error correction, Packet loss, Streaming media",
author = "Eymen Kurdoglu and Yong Liu and Yao Wang",
year = "2017",
month = "12",
day = "7",
doi = "10.1109/TMM.2017.2781362",
language = "English (US)",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Perceptual Quality Maximization for Video Calls with Packet Losses by Optimizing FEC, Frame Rate and Quantization

AU - Kurdoglu, Eymen

AU - Liu, Yong

AU - Wang, Yao

PY - 2017/12/7

Y1 - 2017/12/7

N2 - We consider video calls affected by bursty packet losses, where FEC is applied on a per-frame basis due to the extremely tight delay constraints. In this scenario, a high frame rate at low bitrates leads to large quantization step-sizes (QS), small frames and suboptimal FEC, while a low frame rate at high video bitrates reduces the perceptual quality. At the receiver, damaged frames and others predicted from them are typically discarded, lowering the decoded frame rate. To mitigate frame losses and the following freezing events, temporal layering via hierarchical-P coding structure (hierP) can be used at the cost of lower coding efficiency compared to the traditional IPPP coding structure. In this paper, we study the received video quality maximization for both hierP and IPPP by jointly optimizing the encoding frame rate, QS and the FEC redundancy rates, under the sending bitrate constraint. Building upon Q-STAR, a perceptual video quality model that depends on QS and the decoded frame rate, along with R-STAR, a video bitrate model that depends on QS and the encoding frame rate, we cast the problem as a combinatorial optimization problem. We then solve for the encoding frame rate and the video bitrate using exhaustive search and hill-climbing, as well as a greedy FEC packet distribution algorithm to determine the FEC redundancy rate for each frame. We show that, for independent and identically distributed random losses, (a) the FEC bitrate ratio is an affine function of the packet loss rate, (b) low encoding frame rates are preferred at wider sending bitrate ranges with more severe packet losses, (c) layers are protected more evenly at higher bitrates, and (d) IPPP, while achieving higher Q-STAR scores, is prone to abrupt freezing that is not considered by the Q-STAR model. For bursty losses, we show that (a) layer redundancies are much higher, rising with the mean burst length and reaching up to 80%, (b) hierP achieves higher Q-STAR scores than IPPP in case of longer bursts, and (c) the mean and the variance of decoded frame distances are significantly smaller with hierP.

AB - We consider video calls affected by bursty packet losses, where FEC is applied on a per-frame basis due to the extremely tight delay constraints. In this scenario, a high frame rate at low bitrates leads to large quantization step-sizes (QS), small frames and suboptimal FEC, while a low frame rate at high video bitrates reduces the perceptual quality. At the receiver, damaged frames and others predicted from them are typically discarded, lowering the decoded frame rate. To mitigate frame losses and the following freezing events, temporal layering via hierarchical-P coding structure (hierP) can be used at the cost of lower coding efficiency compared to the traditional IPPP coding structure. In this paper, we study the received video quality maximization for both hierP and IPPP by jointly optimizing the encoding frame rate, QS and the FEC redundancy rates, under the sending bitrate constraint. Building upon Q-STAR, a perceptual video quality model that depends on QS and the decoded frame rate, along with R-STAR, a video bitrate model that depends on QS and the encoding frame rate, we cast the problem as a combinatorial optimization problem. We then solve for the encoding frame rate and the video bitrate using exhaustive search and hill-climbing, as well as a greedy FEC packet distribution algorithm to determine the FEC redundancy rate for each frame. We show that, for independent and identically distributed random losses, (a) the FEC bitrate ratio is an affine function of the packet loss rate, (b) low encoding frame rates are preferred at wider sending bitrate ranges with more severe packet losses, (c) layers are protected more evenly at higher bitrates, and (d) IPPP, while achieving higher Q-STAR scores, is prone to abrupt freezing that is not considered by the Q-STAR model. For bursty losses, we show that (a) layer redundancies are much higher, rising with the mean burst length and reaching up to 80%, (b) hierP achieves higher Q-STAR scores than IPPP in case of longer bursts, and (c) the mean and the variance of decoded frame distances are significantly smaller with hierP.

KW - Bit rate

KW - Delays

KW - Encoding

KW - Forward error correction

KW - Packet loss

KW - Streaming media

UR - http://www.scopus.com/inward/record.url?scp=85038357785&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038357785&partnerID=8YFLogxK

U2 - 10.1109/TMM.2017.2781362

DO - 10.1109/TMM.2017.2781362

M3 - Article

AN - SCOPUS:85038357785

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

ER -