CASENet: Deep category-aware semantic edge detection

Zhiding Yu, Chen Feng, Ming Yu Liu, Srikumar Ramalingam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Boundary and edge cues are highly beneficial in improving a wide variety of vision tasks such as semantic segmentation, object recognition, stereo, and object proposal generation. Recently, the problem of edge detection has been revisited and significant progress has been made with deep learning. While classical edge detection is a challenging binary problem in itself, the category-aware semantic edge detection by nature is an even more challenging multi-label problem. We model the problem such that each edge pixel can be associated with more than one class as they appear in contours or junctions belonging to two or more semantic classes. To this end, we propose a novel end-to-end deep semantic edge learning architecture based on ResNet and a new skip-layer architecture where category-wise edge activations at the top convolution layer share and are fused with the same set of bottom layer features. We then propose a multi-label loss function to supervise the fused activations. We show that our proposed architecture benefits this problem with better performance, and we outperform the current state-of-the-art semantic edge detection methods by a large margin on standard data sets such as SBD and Cityscapes.

Original languageEnglish (US)
Title of host publicationProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1761-1770
Number of pages10
Volume2017-January
ISBN (Electronic)9781538604571
DOIs
StatePublished - Nov 6 2017
Event30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States
Duration: Jul 21 2017Jul 26 2017

Other

Other30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
CountryUnited States
CityHonolulu
Period7/21/177/26/17

Fingerprint

Edge detection
Semantics
Labels
Chemical activation
Object recognition
Convolution
Pixels

ASJC Scopus subject areas

  • Signal Processing
  • Computer Vision and Pattern Recognition

Cite this

Yu, Z., Feng, C., Liu, M. Y., & Ramalingam, S. (2017). CASENet: Deep category-aware semantic edge detection. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (Vol. 2017-January, pp. 1761-1770). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CVPR.2017.191

CASENet : Deep category-aware semantic edge detection. / Yu, Zhiding; Feng, Chen; Liu, Ming Yu; Ramalingam, Srikumar.

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Vol. 2017-January Institute of Electrical and Electronics Engineers Inc., 2017. p. 1761-1770.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yu, Z, Feng, C, Liu, MY & Ramalingam, S 2017, CASENet: Deep category-aware semantic edge detection. in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. vol. 2017-January, Institute of Electrical and Electronics Engineers Inc., pp. 1761-1770, 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, United States, 7/21/17. https://doi.org/10.1109/CVPR.2017.191
Yu Z, Feng C, Liu MY, Ramalingam S. CASENet: Deep category-aware semantic edge detection. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Vol. 2017-January. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1761-1770 https://doi.org/10.1109/CVPR.2017.191
Yu, Zhiding ; Feng, Chen ; Liu, Ming Yu ; Ramalingam, Srikumar. / CASENet : Deep category-aware semantic edge detection. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Vol. 2017-January Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1761-1770
@inproceedings{cbad82282e68460092be75e01a0da483,
title = "CASENet: Deep category-aware semantic edge detection",
abstract = "Boundary and edge cues are highly beneficial in improving a wide variety of vision tasks such as semantic segmentation, object recognition, stereo, and object proposal generation. Recently, the problem of edge detection has been revisited and significant progress has been made with deep learning. While classical edge detection is a challenging binary problem in itself, the category-aware semantic edge detection by nature is an even more challenging multi-label problem. We model the problem such that each edge pixel can be associated with more than one class as they appear in contours or junctions belonging to two or more semantic classes. To this end, we propose a novel end-to-end deep semantic edge learning architecture based on ResNet and a new skip-layer architecture where category-wise edge activations at the top convolution layer share and are fused with the same set of bottom layer features. We then propose a multi-label loss function to supervise the fused activations. We show that our proposed architecture benefits this problem with better performance, and we outperform the current state-of-the-art semantic edge detection methods by a large margin on standard data sets such as SBD and Cityscapes.",
author = "Zhiding Yu and Chen Feng and Liu, {Ming Yu} and Srikumar Ramalingam",
year = "2017",
month = "11",
day = "6",
doi = "10.1109/CVPR.2017.191",
language = "English (US)",
volume = "2017-January",
pages = "1761--1770",
booktitle = "Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - CASENet

T2 - Deep category-aware semantic edge detection

AU - Yu, Zhiding

AU - Feng, Chen

AU - Liu, Ming Yu

AU - Ramalingam, Srikumar

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Boundary and edge cues are highly beneficial in improving a wide variety of vision tasks such as semantic segmentation, object recognition, stereo, and object proposal generation. Recently, the problem of edge detection has been revisited and significant progress has been made with deep learning. While classical edge detection is a challenging binary problem in itself, the category-aware semantic edge detection by nature is an even more challenging multi-label problem. We model the problem such that each edge pixel can be associated with more than one class as they appear in contours or junctions belonging to two or more semantic classes. To this end, we propose a novel end-to-end deep semantic edge learning architecture based on ResNet and a new skip-layer architecture where category-wise edge activations at the top convolution layer share and are fused with the same set of bottom layer features. We then propose a multi-label loss function to supervise the fused activations. We show that our proposed architecture benefits this problem with better performance, and we outperform the current state-of-the-art semantic edge detection methods by a large margin on standard data sets such as SBD and Cityscapes.

AB - Boundary and edge cues are highly beneficial in improving a wide variety of vision tasks such as semantic segmentation, object recognition, stereo, and object proposal generation. Recently, the problem of edge detection has been revisited and significant progress has been made with deep learning. While classical edge detection is a challenging binary problem in itself, the category-aware semantic edge detection by nature is an even more challenging multi-label problem. We model the problem such that each edge pixel can be associated with more than one class as they appear in contours or junctions belonging to two or more semantic classes. To this end, we propose a novel end-to-end deep semantic edge learning architecture based on ResNet and a new skip-layer architecture where category-wise edge activations at the top convolution layer share and are fused with the same set of bottom layer features. We then propose a multi-label loss function to supervise the fused activations. We show that our proposed architecture benefits this problem with better performance, and we outperform the current state-of-the-art semantic edge detection methods by a large margin on standard data sets such as SBD and Cityscapes.

UR - http://www.scopus.com/inward/record.url?scp=85044304093&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044304093&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2017.191

DO - 10.1109/CVPR.2017.191

M3 - Conference contribution

AN - SCOPUS:85044304093

VL - 2017-January

SP - 1761

EP - 1770

BT - Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -