Learning long-range vision for autonomous off-road driving

Raia Hadsell, Pierre Sermanet, Jan Ben, Ayse Erkan, Marco Scoffier, Koray Kavukcuoglu, Urs Muller, Yann LeCun

Research output: Contribution to journalArticle

Abstract

Most vision-based approaches to mobile robotics suffer from the limitations imposed by stereo obstacle detection, which is short range and prone to failure. We present a self-supervised learning process for long-range vision that is able to accurately classify complex terrain at distances up to the horizon, thus allowing superior strategic planning. The success of the learning process is due to the self-supervised training data that are generated on every frame: robust, visually consistent labels from a stereo module; normalized wide-context input windows; and a discriminative and concise feature representation. A deep hierarchical network is trained to extract informative and meaningful features from an input image, and the features are used to train a real-time classifier to predict traversability. The trained classifier sees obstacles and paths from 5 to more than 100 m, far beyond the maximum stereo range of 12 m, and adapts very quickly to new environments. The process was developed and tested on the LAGR (Learning Applied to Ground Robots) mobile robot. Results from a ground truth data set, as well as field test results, are given.

Original languageEnglish (US)
Pages (from-to)120-144
Number of pages25
JournalJournal of Field Robotics
Volume26
Issue number2
DOIs
StatePublished - 2009

Fingerprint

Classifiers
Strategic planning
Supervised learning
Mobile robots
Labels
Robotics
Robots

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications

Cite this

Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., ... LeCun, Y. (2009). Learning long-range vision for autonomous off-road driving. Journal of Field Robotics, 26(2), 120-144. https://doi.org/10.1002/rob.20276

Learning long-range vision for autonomous off-road driving. / Hadsell, Raia; Sermanet, Pierre; Ben, Jan; Erkan, Ayse; Scoffier, Marco; Kavukcuoglu, Koray; Muller, Urs; LeCun, Yann.

In: Journal of Field Robotics, Vol. 26, No. 2, 2009, p. 120-144.

Research output: Contribution to journalArticle

Hadsell, R, Sermanet, P, Ben, J, Erkan, A, Scoffier, M, Kavukcuoglu, K, Muller, U & LeCun, Y 2009, 'Learning long-range vision for autonomous off-road driving', Journal of Field Robotics, vol. 26, no. 2, pp. 120-144. https://doi.org/10.1002/rob.20276
Hadsell R, Sermanet P, Ben J, Erkan A, Scoffier M, Kavukcuoglu K et al. Learning long-range vision for autonomous off-road driving. Journal of Field Robotics. 2009;26(2):120-144. https://doi.org/10.1002/rob.20276
Hadsell, Raia ; Sermanet, Pierre ; Ben, Jan ; Erkan, Ayse ; Scoffier, Marco ; Kavukcuoglu, Koray ; Muller, Urs ; LeCun, Yann. / Learning long-range vision for autonomous off-road driving. In: Journal of Field Robotics. 2009 ; Vol. 26, No. 2. pp. 120-144.
@article{708aa5529eec401a8bfce52a687ff0d4,
title = "Learning long-range vision for autonomous off-road driving",
abstract = "Most vision-based approaches to mobile robotics suffer from the limitations imposed by stereo obstacle detection, which is short range and prone to failure. We present a self-supervised learning process for long-range vision that is able to accurately classify complex terrain at distances up to the horizon, thus allowing superior strategic planning. The success of the learning process is due to the self-supervised training data that are generated on every frame: robust, visually consistent labels from a stereo module; normalized wide-context input windows; and a discriminative and concise feature representation. A deep hierarchical network is trained to extract informative and meaningful features from an input image, and the features are used to train a real-time classifier to predict traversability. The trained classifier sees obstacles and paths from 5 to more than 100 m, far beyond the maximum stereo range of 12 m, and adapts very quickly to new environments. The process was developed and tested on the LAGR (Learning Applied to Ground Robots) mobile robot. Results from a ground truth data set, as well as field test results, are given.",
author = "Raia Hadsell and Pierre Sermanet and Jan Ben and Ayse Erkan and Marco Scoffier and Koray Kavukcuoglu and Urs Muller and Yann LeCun",
year = "2009",
doi = "10.1002/rob.20276",
language = "English (US)",
volume = "26",
pages = "120--144",
journal = "Journal of Field Robotics",
issn = "1556-4959",
publisher = "John Wiley and Sons Inc.",
number = "2",

}

TY - JOUR

T1 - Learning long-range vision for autonomous off-road driving

AU - Hadsell, Raia

AU - Sermanet, Pierre

AU - Ben, Jan

AU - Erkan, Ayse

AU - Scoffier, Marco

AU - Kavukcuoglu, Koray

AU - Muller, Urs

AU - LeCun, Yann

PY - 2009

Y1 - 2009

N2 - Most vision-based approaches to mobile robotics suffer from the limitations imposed by stereo obstacle detection, which is short range and prone to failure. We present a self-supervised learning process for long-range vision that is able to accurately classify complex terrain at distances up to the horizon, thus allowing superior strategic planning. The success of the learning process is due to the self-supervised training data that are generated on every frame: robust, visually consistent labels from a stereo module; normalized wide-context input windows; and a discriminative and concise feature representation. A deep hierarchical network is trained to extract informative and meaningful features from an input image, and the features are used to train a real-time classifier to predict traversability. The trained classifier sees obstacles and paths from 5 to more than 100 m, far beyond the maximum stereo range of 12 m, and adapts very quickly to new environments. The process was developed and tested on the LAGR (Learning Applied to Ground Robots) mobile robot. Results from a ground truth data set, as well as field test results, are given.

AB - Most vision-based approaches to mobile robotics suffer from the limitations imposed by stereo obstacle detection, which is short range and prone to failure. We present a self-supervised learning process for long-range vision that is able to accurately classify complex terrain at distances up to the horizon, thus allowing superior strategic planning. The success of the learning process is due to the self-supervised training data that are generated on every frame: robust, visually consistent labels from a stereo module; normalized wide-context input windows; and a discriminative and concise feature representation. A deep hierarchical network is trained to extract informative and meaningful features from an input image, and the features are used to train a real-time classifier to predict traversability. The trained classifier sees obstacles and paths from 5 to more than 100 m, far beyond the maximum stereo range of 12 m, and adapts very quickly to new environments. The process was developed and tested on the LAGR (Learning Applied to Ground Robots) mobile robot. Results from a ground truth data set, as well as field test results, are given.

UR - http://www.scopus.com/inward/record.url?scp=67649219352&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67649219352&partnerID=8YFLogxK

U2 - 10.1002/rob.20276

DO - 10.1002/rob.20276

M3 - Article

VL - 26

SP - 120

EP - 144

JO - Journal of Field Robotics

JF - Journal of Field Robotics

SN - 1556-4959

IS - 2

ER -