Saliency inspired full-reference quality metrics for packet-loss-impaired video

Xin Feng, Tao Liu, Dan Yang, Yao Wang

Research output: Contribution to journalArticle

Abstract

This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti's saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.

Original languageEnglish (US)
Article number5665781
Pages (from-to)81-88
Number of pages8
JournalIEEE Transactions on Broadcasting
Volume57
Issue number1
DOIs
StatePublished - Mar 2011

Fingerprint

Packet loss
Pixels
Linear regression
Regression analysis

Keywords

  • Attention change
  • Packet loss distortion
  • Saliency
  • Video quality assessment

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology

Cite this

Saliency inspired full-reference quality metrics for packet-loss-impaired video. / Feng, Xin; Liu, Tao; Yang, Dan; Wang, Yao.

In: IEEE Transactions on Broadcasting, Vol. 57, No. 1, 5665781, 03.2011, p. 81-88.

Research output: Contribution to journalArticle

@article{2661edf090f24526b9e75757ee768867,
title = "Saliency inspired full-reference quality metrics for packet-loss-impaired video",
abstract = "This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti's saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.",
keywords = "Attention change, Packet loss distortion, Saliency, Video quality assessment",
author = "Xin Feng and Tao Liu and Dan Yang and Yao Wang",
year = "2011",
month = "3",
doi = "10.1109/TBC.2010.2092150",
language = "English (US)",
volume = "57",
pages = "81--88",
journal = "IEEE Transactions on Broadcasting",
issn = "0018-9316",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

TY - JOUR

T1 - Saliency inspired full-reference quality metrics for packet-loss-impaired video

AU - Feng, Xin

AU - Liu, Tao

AU - Yang, Dan

AU - Wang, Yao

PY - 2011/3

Y1 - 2011/3

N2 - This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti's saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.

AB - This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti's saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.

KW - Attention change

KW - Packet loss distortion

KW - Saliency

KW - Video quality assessment

UR - http://www.scopus.com/inward/record.url?scp=79952003635&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79952003635&partnerID=8YFLogxK

U2 - 10.1109/TBC.2010.2092150

DO - 10.1109/TBC.2010.2092150

M3 - Article

AN - SCOPUS:79952003635

VL - 57

SP - 81

EP - 88

JO - IEEE Transactions on Broadcasting

JF - IEEE Transactions on Broadcasting

SN - 0018-9316

IS - 1

M1 - 5665781

ER -