Instance segmentation of indoor scenes using a coverage loss

Nathan Silberman, David Sontag, Robert Fergus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A major limitation of existing models for semantic segmentation is the inability to identify individual instances of the same class: when labeling pixels with only semantic classes, a set of pixels with the same label could represent a single object or ten. In this work, we introduce a model to perform both semantic and instance segmentation simultaneously. We introduce a new higher-order loss function that directly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network. We apply our model to the NYU Depth V2 dataset, obtaining state of the art results.

Original languageEnglish (US)
Title of host publicationComputer Vision, ECCV 2014 - 13th European Conference, Proceedings
PublisherSpringer Verlag
Pages616-631
Number of pages16
Volume8689 LNCS
EditionPART 1
ISBN (Print)9783319105895
DOIs
StatePublished - 2014
Event13th European Conference on Computer Vision, ECCV 2014 - Zurich, Switzerland
Duration: Sep 6 2014Sep 12 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume8689 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other13th European Conference on Computer Vision, ECCV 2014
CountrySwitzerland
CityZurich
Period9/6/149/12/14

Fingerprint

Coverage
Segmentation
Semantics
Pixel
Pixels
Loss Function
Labeling
Labels
Model
Higher Order
Minimise
Metric
Evaluate
Class
Object

Keywords

  • Deep Learning
  • Semantic Segmentation

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Silberman, N., Sontag, D., & Fergus, R. (2014). Instance segmentation of indoor scenes using a coverage loss. In Computer Vision, ECCV 2014 - 13th European Conference, Proceedings (PART 1 ed., Vol. 8689 LNCS, pp. 616-631). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8689 LNCS, No. PART 1). Springer Verlag. https://doi.org/10.1007/978-3-319-10590-1_40

Instance segmentation of indoor scenes using a coverage loss. / Silberman, Nathan; Sontag, David; Fergus, Robert.

Computer Vision, ECCV 2014 - 13th European Conference, Proceedings. Vol. 8689 LNCS PART 1. ed. Springer Verlag, 2014. p. 616-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8689 LNCS, No. PART 1).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Silberman, N, Sontag, D & Fergus, R 2014, Instance segmentation of indoor scenes using a coverage loss. in Computer Vision, ECCV 2014 - 13th European Conference, Proceedings. PART 1 edn, vol. 8689 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 8689 LNCS, Springer Verlag, pp. 616-631, 13th European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 9/6/14. https://doi.org/10.1007/978-3-319-10590-1_40
Silberman N, Sontag D, Fergus R. Instance segmentation of indoor scenes using a coverage loss. In Computer Vision, ECCV 2014 - 13th European Conference, Proceedings. PART 1 ed. Vol. 8689 LNCS. Springer Verlag. 2014. p. 616-631. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). https://doi.org/10.1007/978-3-319-10590-1_40
Silberman, Nathan ; Sontag, David ; Fergus, Robert. / Instance segmentation of indoor scenes using a coverage loss. Computer Vision, ECCV 2014 - 13th European Conference, Proceedings. Vol. 8689 LNCS PART 1. ed. Springer Verlag, 2014. pp. 616-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).
@inproceedings{32d20fe5694a4ebc8bed17819615697c,
title = "Instance segmentation of indoor scenes using a coverage loss",
abstract = "A major limitation of existing models for semantic segmentation is the inability to identify individual instances of the same class: when labeling pixels with only semantic classes, a set of pixels with the same label could represent a single object or ten. In this work, we introduce a model to perform both semantic and instance segmentation simultaneously. We introduce a new higher-order loss function that directly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network. We apply our model to the NYU Depth V2 dataset, obtaining state of the art results.",
keywords = "Deep Learning, Semantic Segmentation",
author = "Nathan Silberman and David Sontag and Robert Fergus",
year = "2014",
doi = "10.1007/978-3-319-10590-1_40",
language = "English (US)",
isbn = "9783319105895",
volume = "8689 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
number = "PART 1",
pages = "616--631",
booktitle = "Computer Vision, ECCV 2014 - 13th European Conference, Proceedings",
edition = "PART 1",

}

TY - GEN

T1 - Instance segmentation of indoor scenes using a coverage loss

AU - Silberman, Nathan

AU - Sontag, David

AU - Fergus, Robert

PY - 2014

Y1 - 2014

N2 - A major limitation of existing models for semantic segmentation is the inability to identify individual instances of the same class: when labeling pixels with only semantic classes, a set of pixels with the same label could represent a single object or ten. In this work, we introduce a model to perform both semantic and instance segmentation simultaneously. We introduce a new higher-order loss function that directly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network. We apply our model to the NYU Depth V2 dataset, obtaining state of the art results.

AB - A major limitation of existing models for semantic segmentation is the inability to identify individual instances of the same class: when labeling pixels with only semantic classes, a set of pixels with the same label could represent a single object or ten. In this work, we introduce a model to perform both semantic and instance segmentation simultaneously. We introduce a new higher-order loss function that directly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network. We apply our model to the NYU Depth V2 dataset, obtaining state of the art results.

KW - Deep Learning

KW - Semantic Segmentation

UR - http://www.scopus.com/inward/record.url?scp=84906507967&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906507967&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-10590-1_40

DO - 10.1007/978-3-319-10590-1_40

M3 - Conference contribution

SN - 9783319105895

VL - 8689 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 616

EP - 631

BT - Computer Vision, ECCV 2014 - 13th European Conference, Proceedings

PB - Springer Verlag

ER -