Foreground-Background Separation from Video Clips via Motion-Assisted Matrix Restoration

Xinchen Ye, Jingyu Yang, Xin Sun, Kun Li, Chunping Hou, Yao Wang

Research output: Contribution to journalArticle

Abstract

Separation of video clips into foreground and background components is a useful and important technique, making recognition, classification, and scene analysis more efficient. In this paper, we propose a motion-assisted matrix restoration (MAMR) model for foreground-background separation in video clips. In the proposed MAMR model, the backgrounds across frames are modeled by a low-rank matrix, while the foreground objects are modeled by a sparse matrix. To facilitate efficient foreground-background separation, a dense motion field is estimated for each frame, and mapped into a weighting matrix which indicates the likelihood that each pixel belongs to the background. Anchor frames are selected in the dense motion estimation to overcome the difficulty of detecting slowly moving objects and camouflages. In addition, we extend our model to a robust MAMR model against noise for practical applications. Evaluations on challenging datasets demonstrate that our method outperforms many other state-of-the-art methods, and is versatile for a wide range of surveillance videos.

Original languageEnglish (US)
Article number7014298
Pages (from-to)1721-1734
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume25
Issue number11
DOIs
StatePublished - Nov 1 2015

Fingerprint

Restoration
Camouflage
Motion estimation
Anchors
Pixels

Keywords

  • Background segmentation/subtraction
  • matrix restoration
  • motion detection
  • optical flow
  • video surveillance

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology

Cite this

Foreground-Background Separation from Video Clips via Motion-Assisted Matrix Restoration. / Ye, Xinchen; Yang, Jingyu; Sun, Xin; Li, Kun; Hou, Chunping; Wang, Yao.

In: IEEE Transactions on Circuits and Systems for Video Technology, Vol. 25, No. 11, 7014298, 01.11.2015, p. 1721-1734.

Research output: Contribution to journalArticle

Ye, Xinchen ; Yang, Jingyu ; Sun, Xin ; Li, Kun ; Hou, Chunping ; Wang, Yao. / Foreground-Background Separation from Video Clips via Motion-Assisted Matrix Restoration. In: IEEE Transactions on Circuits and Systems for Video Technology. 2015 ; Vol. 25, No. 11. pp. 1721-1734.
@article{8a8b3e3ade4a43178711e5cf5daf1481,
title = "Foreground-Background Separation from Video Clips via Motion-Assisted Matrix Restoration",
abstract = "Separation of video clips into foreground and background components is a useful and important technique, making recognition, classification, and scene analysis more efficient. In this paper, we propose a motion-assisted matrix restoration (MAMR) model for foreground-background separation in video clips. In the proposed MAMR model, the backgrounds across frames are modeled by a low-rank matrix, while the foreground objects are modeled by a sparse matrix. To facilitate efficient foreground-background separation, a dense motion field is estimated for each frame, and mapped into a weighting matrix which indicates the likelihood that each pixel belongs to the background. Anchor frames are selected in the dense motion estimation to overcome the difficulty of detecting slowly moving objects and camouflages. In addition, we extend our model to a robust MAMR model against noise for practical applications. Evaluations on challenging datasets demonstrate that our method outperforms many other state-of-the-art methods, and is versatile for a wide range of surveillance videos.",
keywords = "Background segmentation/subtraction, matrix restoration, motion detection, optical flow, video surveillance",
author = "Xinchen Ye and Jingyu Yang and Xin Sun and Kun Li and Chunping Hou and Yao Wang",
year = "2015",
month = "11",
day = "1",
doi = "10.1109/TCSVT.2015.2392491",
language = "English (US)",
volume = "25",
pages = "1721--1734",
journal = "IEEE Transactions on Circuits and Systems for Video Technology",
issn = "1051-8215",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "11",

}

TY - JOUR

T1 - Foreground-Background Separation from Video Clips via Motion-Assisted Matrix Restoration

AU - Ye, Xinchen

AU - Yang, Jingyu

AU - Sun, Xin

AU - Li, Kun

AU - Hou, Chunping

AU - Wang, Yao

PY - 2015/11/1

Y1 - 2015/11/1

N2 - Separation of video clips into foreground and background components is a useful and important technique, making recognition, classification, and scene analysis more efficient. In this paper, we propose a motion-assisted matrix restoration (MAMR) model for foreground-background separation in video clips. In the proposed MAMR model, the backgrounds across frames are modeled by a low-rank matrix, while the foreground objects are modeled by a sparse matrix. To facilitate efficient foreground-background separation, a dense motion field is estimated for each frame, and mapped into a weighting matrix which indicates the likelihood that each pixel belongs to the background. Anchor frames are selected in the dense motion estimation to overcome the difficulty of detecting slowly moving objects and camouflages. In addition, we extend our model to a robust MAMR model against noise for practical applications. Evaluations on challenging datasets demonstrate that our method outperforms many other state-of-the-art methods, and is versatile for a wide range of surveillance videos.

AB - Separation of video clips into foreground and background components is a useful and important technique, making recognition, classification, and scene analysis more efficient. In this paper, we propose a motion-assisted matrix restoration (MAMR) model for foreground-background separation in video clips. In the proposed MAMR model, the backgrounds across frames are modeled by a low-rank matrix, while the foreground objects are modeled by a sparse matrix. To facilitate efficient foreground-background separation, a dense motion field is estimated for each frame, and mapped into a weighting matrix which indicates the likelihood that each pixel belongs to the background. Anchor frames are selected in the dense motion estimation to overcome the difficulty of detecting slowly moving objects and camouflages. In addition, we extend our model to a robust MAMR model against noise for practical applications. Evaluations on challenging datasets demonstrate that our method outperforms many other state-of-the-art methods, and is versatile for a wide range of surveillance videos.

KW - Background segmentation/subtraction

KW - matrix restoration

KW - motion detection

KW - optical flow

KW - video surveillance

UR - http://www.scopus.com/inward/record.url?scp=84960474011&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960474011&partnerID=8YFLogxK

U2 - 10.1109/TCSVT.2015.2392491

DO - 10.1109/TCSVT.2015.2392491

M3 - Article

VL - 25

SP - 1721

EP - 1734

JO - IEEE Transactions on Circuits and Systems for Video Technology

JF - IEEE Transactions on Circuits and Systems for Video Technology

SN - 1051-8215

IS - 11

M1 - 7014298

ER -