Through-wall object recognition and pose estimation

Ruoyu Wang, Siyuan Xiang, Chen Feng, Pu Wang, Semiha Ergan, Yi Fang

Research output: Contribution to conferencePaper

Abstract

Robots need to perceive beyond lines of sight, e.g., to avoid cutting water pipes or electric wires when drilling holes on a wall. Recent off-the-shelf radio frequency (RF) imaging sensors ease the process of 3D sensing inside or through walls. Yet unlike optical images, RF images are difficult to understand by a human. Meanwhile, in practice, RF components are often subject to hardware imperfections, resulting in distorted RF images, whose quality could be far from the claimed specifications. Thus, we introduce several challenging geometric and semantic perception tasks on such signals, including object and material recognition, fine-grained property classification and pose estimation. Since detailed forward modeling of such sensors is sometimes difficult, due to hidden or inaccessible system parameters, onboard processing procedures and limited access to raw RF waveform, we tackled the above tasks by supervised machine learning. We collected a large dataset of RF images of utility objects through a mock wall as the input of our algorithm, and the corresponding optical images were taken from the other side of the wall simultaneously as the ground truth. We designed three learning algorithms based on nearest neighbors or neural networks, and report their performances on the dataset. Our experiments showed reasonable results for semantic perception tasks yet unsatisfactory results for geometric ones, calling for more efforts in this research direction.

Original languageEnglish (US)
Pages1176-1183
Number of pages8
StatePublished - Jan 1 2019
Event36th International Symposium on Automation and Robotics in Construction, ISARC 2019 - Banff, Canada
Duration: May 21 2019May 24 2019

Conference

Conference36th International Symposium on Automation and Robotics in Construction, ISARC 2019
CountryCanada
CityBanff
Period5/21/195/24/19

Fingerprint

Object recognition
Electric wire
Semantics
Sensors
Learning algorithms
Image quality
Learning systems
Drilling
Pipe
Robots
Neural networks
Specifications
Hardware
Imaging techniques
Defects
Processing
Water
Experiments

Keywords

  • Deep learning
  • Object recognition
  • Pose estimation
  • Through-wall imaging

ASJC Scopus subject areas

  • Artificial Intelligence
  • Building and Construction
  • Human-Computer Interaction

Cite this

Wang, R., Xiang, S., Feng, C., Wang, P., Ergan, S., & Fang, Y. (2019). Through-wall object recognition and pose estimation. 1176-1183. Paper presented at 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, Canada.

Through-wall object recognition and pose estimation. / Wang, Ruoyu; Xiang, Siyuan; Feng, Chen; Wang, Pu; Ergan, Semiha; Fang, Yi.

2019. 1176-1183 Paper presented at 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, Canada.

Research output: Contribution to conferencePaper

Wang, R, Xiang, S, Feng, C, Wang, P, Ergan, S & Fang, Y 2019, 'Through-wall object recognition and pose estimation', Paper presented at 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, Canada, 5/21/19 - 5/24/19 pp. 1176-1183.
Wang R, Xiang S, Feng C, Wang P, Ergan S, Fang Y. Through-wall object recognition and pose estimation. 2019. Paper presented at 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, Canada.
Wang, Ruoyu ; Xiang, Siyuan ; Feng, Chen ; Wang, Pu ; Ergan, Semiha ; Fang, Yi. / Through-wall object recognition and pose estimation. Paper presented at 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, Canada.8 p.
@conference{8f30e51db7c54293908d855033c85dae,
title = "Through-wall object recognition and pose estimation",
abstract = "Robots need to perceive beyond lines of sight, e.g., to avoid cutting water pipes or electric wires when drilling holes on a wall. Recent off-the-shelf radio frequency (RF) imaging sensors ease the process of 3D sensing inside or through walls. Yet unlike optical images, RF images are difficult to understand by a human. Meanwhile, in practice, RF components are often subject to hardware imperfections, resulting in distorted RF images, whose quality could be far from the claimed specifications. Thus, we introduce several challenging geometric and semantic perception tasks on such signals, including object and material recognition, fine-grained property classification and pose estimation. Since detailed forward modeling of such sensors is sometimes difficult, due to hidden or inaccessible system parameters, onboard processing procedures and limited access to raw RF waveform, we tackled the above tasks by supervised machine learning. We collected a large dataset of RF images of utility objects through a mock wall as the input of our algorithm, and the corresponding optical images were taken from the other side of the wall simultaneously as the ground truth. We designed three learning algorithms based on nearest neighbors or neural networks, and report their performances on the dataset. Our experiments showed reasonable results for semantic perception tasks yet unsatisfactory results for geometric ones, calling for more efforts in this research direction.",
keywords = "Deep learning, Object recognition, Pose estimation, Through-wall imaging",
author = "Ruoyu Wang and Siyuan Xiang and Chen Feng and Pu Wang and Semiha Ergan and Yi Fang",
year = "2019",
month = "1",
day = "1",
language = "English (US)",
pages = "1176--1183",
note = "36th International Symposium on Automation and Robotics in Construction, ISARC 2019 ; Conference date: 21-05-2019 Through 24-05-2019",

}

TY - CONF

T1 - Through-wall object recognition and pose estimation

AU - Wang, Ruoyu

AU - Xiang, Siyuan

AU - Feng, Chen

AU - Wang, Pu

AU - Ergan, Semiha

AU - Fang, Yi

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Robots need to perceive beyond lines of sight, e.g., to avoid cutting water pipes or electric wires when drilling holes on a wall. Recent off-the-shelf radio frequency (RF) imaging sensors ease the process of 3D sensing inside or through walls. Yet unlike optical images, RF images are difficult to understand by a human. Meanwhile, in practice, RF components are often subject to hardware imperfections, resulting in distorted RF images, whose quality could be far from the claimed specifications. Thus, we introduce several challenging geometric and semantic perception tasks on such signals, including object and material recognition, fine-grained property classification and pose estimation. Since detailed forward modeling of such sensors is sometimes difficult, due to hidden or inaccessible system parameters, onboard processing procedures and limited access to raw RF waveform, we tackled the above tasks by supervised machine learning. We collected a large dataset of RF images of utility objects through a mock wall as the input of our algorithm, and the corresponding optical images were taken from the other side of the wall simultaneously as the ground truth. We designed three learning algorithms based on nearest neighbors or neural networks, and report their performances on the dataset. Our experiments showed reasonable results for semantic perception tasks yet unsatisfactory results for geometric ones, calling for more efforts in this research direction.

AB - Robots need to perceive beyond lines of sight, e.g., to avoid cutting water pipes or electric wires when drilling holes on a wall. Recent off-the-shelf radio frequency (RF) imaging sensors ease the process of 3D sensing inside or through walls. Yet unlike optical images, RF images are difficult to understand by a human. Meanwhile, in practice, RF components are often subject to hardware imperfections, resulting in distorted RF images, whose quality could be far from the claimed specifications. Thus, we introduce several challenging geometric and semantic perception tasks on such signals, including object and material recognition, fine-grained property classification and pose estimation. Since detailed forward modeling of such sensors is sometimes difficult, due to hidden or inaccessible system parameters, onboard processing procedures and limited access to raw RF waveform, we tackled the above tasks by supervised machine learning. We collected a large dataset of RF images of utility objects through a mock wall as the input of our algorithm, and the corresponding optical images were taken from the other side of the wall simultaneously as the ground truth. We designed three learning algorithms based on nearest neighbors or neural networks, and report their performances on the dataset. Our experiments showed reasonable results for semantic perception tasks yet unsatisfactory results for geometric ones, calling for more efforts in this research direction.

KW - Deep learning

KW - Object recognition

KW - Pose estimation

KW - Through-wall imaging

UR - http://www.scopus.com/inward/record.url?scp=85071417457&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071417457&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85071417457

SP - 1176

EP - 1183

ER -