Reconfigurable network for efficient inferencing in autonomous vehicles

Shihong Fang, Anna Choromanska

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a reconfigurable network for efficient inference dedicated to autonomous platforms equipped with multiple perception sensors. The size of the network for steering autonomous platforms grows proportionally to the number of installed sensors eventually preventing the usage of multiple sensors in real-time applications due to an inefficient inference. Our approach hinges on the observation that multiple sensors provide a large stream of data, where only a fraction of the data is relevant for the performed task at any given moment in time. The architecture of the reconfigurable network that we propose contains separate feature extractors, called experts, for each sensor. The decisive block of our model is the gating network, which online decides which sensor provides the data that is most relevant for driving. It then reconfigures the network by activating only the relevant expert corresponding to that sensor and deactivating the remaining ones. As a consequence, the model never extracts features from data that are irrelevant for driving. The gating network takes the data from all inputs and thus to avoid explosion of computation time and memory space it has to be realized as a small and shallow network. We verify our model on the unmanned ground vehicle (UGV) comprising of the 1/6 scale remote control truck equipped with three cameras. We demonstrate that the reconfigurable network correctly chooses experts in real-time allowing the reduction of computations cost for the whole model without deteriorating its performance.

Original languageEnglish (US)
Title of host publication2019 International Conference on Robotics and Automation, ICRA 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1183-1189
Number of pages7
ISBN (Electronic)9781538660263
DOIs
StatePublished - May 1 2019
Event2019 International Conference on Robotics and Automation, ICRA 2019 - Montreal, Canada
Duration: May 20 2019May 24 2019

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
Volume2019-May
ISSN (Print)1050-4729

Conference

Conference2019 International Conference on Robotics and Automation, ICRA 2019
CountryCanada
CityMontreal
Period5/20/195/24/19

Fingerprint

Sensors
Unmanned vehicles
Ground vehicles
Hinges
Remote control
Trucks
Explosions
Cameras
Data storage equipment
Costs

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Cite this

Fang, S., & Choromanska, A. (2019). Reconfigurable network for efficient inferencing in autonomous vehicles. In 2019 International Conference on Robotics and Automation, ICRA 2019 (pp. 1183-1189). [8794064] (Proceedings - IEEE International Conference on Robotics and Automation; Vol. 2019-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRA.2019.8794064

Reconfigurable network for efficient inferencing in autonomous vehicles. / Fang, Shihong; Choromanska, Anna.

2019 International Conference on Robotics and Automation, ICRA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1183-1189 8794064 (Proceedings - IEEE International Conference on Robotics and Automation; Vol. 2019-May).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fang, S & Choromanska, A 2019, Reconfigurable network for efficient inferencing in autonomous vehicles. in 2019 International Conference on Robotics and Automation, ICRA 2019., 8794064, Proceedings - IEEE International Conference on Robotics and Automation, vol. 2019-May, Institute of Electrical and Electronics Engineers Inc., pp. 1183-1189, 2019 International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada, 5/20/19. https://doi.org/10.1109/ICRA.2019.8794064
Fang S, Choromanska A. Reconfigurable network for efficient inferencing in autonomous vehicles. In 2019 International Conference on Robotics and Automation, ICRA 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1183-1189. 8794064. (Proceedings - IEEE International Conference on Robotics and Automation). https://doi.org/10.1109/ICRA.2019.8794064
Fang, Shihong ; Choromanska, Anna. / Reconfigurable network for efficient inferencing in autonomous vehicles. 2019 International Conference on Robotics and Automation, ICRA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1183-1189 (Proceedings - IEEE International Conference on Robotics and Automation).
@inproceedings{8b7af733476b4269a615ddb3a8105fa1,
title = "Reconfigurable network for efficient inferencing in autonomous vehicles",
abstract = "We propose a reconfigurable network for efficient inference dedicated to autonomous platforms equipped with multiple perception sensors. The size of the network for steering autonomous platforms grows proportionally to the number of installed sensors eventually preventing the usage of multiple sensors in real-time applications due to an inefficient inference. Our approach hinges on the observation that multiple sensors provide a large stream of data, where only a fraction of the data is relevant for the performed task at any given moment in time. The architecture of the reconfigurable network that we propose contains separate feature extractors, called experts, for each sensor. The decisive block of our model is the gating network, which online decides which sensor provides the data that is most relevant for driving. It then reconfigures the network by activating only the relevant expert corresponding to that sensor and deactivating the remaining ones. As a consequence, the model never extracts features from data that are irrelevant for driving. The gating network takes the data from all inputs and thus to avoid explosion of computation time and memory space it has to be realized as a small and shallow network. We verify our model on the unmanned ground vehicle (UGV) comprising of the 1/6 scale remote control truck equipped with three cameras. We demonstrate that the reconfigurable network correctly chooses experts in real-time allowing the reduction of computations cost for the whole model without deteriorating its performance.",
author = "Shihong Fang and Anna Choromanska",
year = "2019",
month = "5",
day = "1",
doi = "10.1109/ICRA.2019.8794064",
language = "English (US)",
series = "Proceedings - IEEE International Conference on Robotics and Automation",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1183--1189",
booktitle = "2019 International Conference on Robotics and Automation, ICRA 2019",

}

TY - GEN

T1 - Reconfigurable network for efficient inferencing in autonomous vehicles

AU - Fang, Shihong

AU - Choromanska, Anna

PY - 2019/5/1

Y1 - 2019/5/1

N2 - We propose a reconfigurable network for efficient inference dedicated to autonomous platforms equipped with multiple perception sensors. The size of the network for steering autonomous platforms grows proportionally to the number of installed sensors eventually preventing the usage of multiple sensors in real-time applications due to an inefficient inference. Our approach hinges on the observation that multiple sensors provide a large stream of data, where only a fraction of the data is relevant for the performed task at any given moment in time. The architecture of the reconfigurable network that we propose contains separate feature extractors, called experts, for each sensor. The decisive block of our model is the gating network, which online decides which sensor provides the data that is most relevant for driving. It then reconfigures the network by activating only the relevant expert corresponding to that sensor and deactivating the remaining ones. As a consequence, the model never extracts features from data that are irrelevant for driving. The gating network takes the data from all inputs and thus to avoid explosion of computation time and memory space it has to be realized as a small and shallow network. We verify our model on the unmanned ground vehicle (UGV) comprising of the 1/6 scale remote control truck equipped with three cameras. We demonstrate that the reconfigurable network correctly chooses experts in real-time allowing the reduction of computations cost for the whole model without deteriorating its performance.

AB - We propose a reconfigurable network for efficient inference dedicated to autonomous platforms equipped with multiple perception sensors. The size of the network for steering autonomous platforms grows proportionally to the number of installed sensors eventually preventing the usage of multiple sensors in real-time applications due to an inefficient inference. Our approach hinges on the observation that multiple sensors provide a large stream of data, where only a fraction of the data is relevant for the performed task at any given moment in time. The architecture of the reconfigurable network that we propose contains separate feature extractors, called experts, for each sensor. The decisive block of our model is the gating network, which online decides which sensor provides the data that is most relevant for driving. It then reconfigures the network by activating only the relevant expert corresponding to that sensor and deactivating the remaining ones. As a consequence, the model never extracts features from data that are irrelevant for driving. The gating network takes the data from all inputs and thus to avoid explosion of computation time and memory space it has to be realized as a small and shallow network. We verify our model on the unmanned ground vehicle (UGV) comprising of the 1/6 scale remote control truck equipped with three cameras. We demonstrate that the reconfigurable network correctly chooses experts in real-time allowing the reduction of computations cost for the whole model without deteriorating its performance.

UR - http://www.scopus.com/inward/record.url?scp=85071438756&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071438756&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2019.8794064

DO - 10.1109/ICRA.2019.8794064

M3 - Conference contribution

AN - SCOPUS:85071438756

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 1183

EP - 1189

BT - 2019 International Conference on Robotics and Automation, ICRA 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -