Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites

Chen Feng, Yong Xiao, Aaron Willette, Wes McGee, Vineet R. Kamat

Research output: Contribution to journalArticle

Abstract

Unlike robotics in the manufacturing industry, on-site construction robotics has to consider and address two unique challenges: 1) the rugged, evolving, and unstructured environment of typical work sites; and 2) the reversed spatial relationship between the product and the manipulator, i.e., the manipulator has to travel to and localize itself at the work face, rather than a partially complete product arriving at an anchored manipulator. The presented research designed and implemented algorithms that address these challenges and enable autonomous robotic assembly of freeform modular structures on construction sites. Building on the authors' previous work in computer-vision-based pose estimation, the designed algorithms enable a mobile robotic manipulator to: 1) autonomously identify and grasp prismatic building components (e.g., bricks, blocks) that are typically non-unique and arbitrarily stored on-site; and 2) assemble these components into pre-designed modular structures. The algorithms use a single camera and a visual marker-based metrology to rapidly establish local reference frames and to detect staged building components. Based on the design of the structure being assembled, the algorithms automatically determine the assembly sequence. Furthermore, if a 3D camera is mounted on the manipulator, 3D point clouds can be readily captured and registered into a same reference frame through our marker-based metrology and the manipulator's internal encoders, either after construction to facilitate as-built Building Information Model (BIM) generation, or during construction to document details of the progress. Implemented using a 7-axis KUKA KR100 robotic manipulator, the presented robotic system has successfully assembled various structures and created as-built 3D point cloud models autonomously, demonstrating the designed algorithms' effectiveness in autonomous on-site construction robotics applications.

Original languageEnglish (US)
Article number1920
Pages (from-to)128-138
Number of pages11
JournalAutomation in Construction
Volume59
DOIs
StatePublished - Nov 1 2015

Fingerprint

Robotic assembly
Manipulators
Robotics
Scanning
Cameras
Brick
Computer vision

Keywords

  • As-built 3D modeling
  • Autonomous assembly
  • On-site construction robotics
  • Pose estimation

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Civil and Structural Engineering
  • Building and Construction

Cite this

Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites. / Feng, Chen; Xiao, Yong; Willette, Aaron; McGee, Wes; Kamat, Vineet R.

In: Automation in Construction, Vol. 59, 1920, 01.11.2015, p. 128-138.

Research output: Contribution to journalArticle

Feng, Chen ; Xiao, Yong ; Willette, Aaron ; McGee, Wes ; Kamat, Vineet R. / Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites. In: Automation in Construction. 2015 ; Vol. 59. pp. 128-138.
@article{ad7ebad195b6494ab641d314fb2e5175,
title = "Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites",
abstract = "Unlike robotics in the manufacturing industry, on-site construction robotics has to consider and address two unique challenges: 1) the rugged, evolving, and unstructured environment of typical work sites; and 2) the reversed spatial relationship between the product and the manipulator, i.e., the manipulator has to travel to and localize itself at the work face, rather than a partially complete product arriving at an anchored manipulator. The presented research designed and implemented algorithms that address these challenges and enable autonomous robotic assembly of freeform modular structures on construction sites. Building on the authors' previous work in computer-vision-based pose estimation, the designed algorithms enable a mobile robotic manipulator to: 1) autonomously identify and grasp prismatic building components (e.g., bricks, blocks) that are typically non-unique and arbitrarily stored on-site; and 2) assemble these components into pre-designed modular structures. The algorithms use a single camera and a visual marker-based metrology to rapidly establish local reference frames and to detect staged building components. Based on the design of the structure being assembled, the algorithms automatically determine the assembly sequence. Furthermore, if a 3D camera is mounted on the manipulator, 3D point clouds can be readily captured and registered into a same reference frame through our marker-based metrology and the manipulator's internal encoders, either after construction to facilitate as-built Building Information Model (BIM) generation, or during construction to document details of the progress. Implemented using a 7-axis KUKA KR100 robotic manipulator, the presented robotic system has successfully assembled various structures and created as-built 3D point cloud models autonomously, demonstrating the designed algorithms' effectiveness in autonomous on-site construction robotics applications.",
keywords = "As-built 3D modeling, Autonomous assembly, On-site construction robotics, Pose estimation",
author = "Chen Feng and Yong Xiao and Aaron Willette and Wes McGee and Kamat, {Vineet R.}",
year = "2015",
month = "11",
day = "1",
doi = "10.1016/j.autcon.2015.06.002",
language = "English (US)",
volume = "59",
pages = "128--138",
journal = "Automation in Construction",
issn = "0926-5805",
publisher = "Elsevier",

}

TY - JOUR

T1 - Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites

AU - Feng, Chen

AU - Xiao, Yong

AU - Willette, Aaron

AU - McGee, Wes

AU - Kamat, Vineet R.

PY - 2015/11/1

Y1 - 2015/11/1

N2 - Unlike robotics in the manufacturing industry, on-site construction robotics has to consider and address two unique challenges: 1) the rugged, evolving, and unstructured environment of typical work sites; and 2) the reversed spatial relationship between the product and the manipulator, i.e., the manipulator has to travel to and localize itself at the work face, rather than a partially complete product arriving at an anchored manipulator. The presented research designed and implemented algorithms that address these challenges and enable autonomous robotic assembly of freeform modular structures on construction sites. Building on the authors' previous work in computer-vision-based pose estimation, the designed algorithms enable a mobile robotic manipulator to: 1) autonomously identify and grasp prismatic building components (e.g., bricks, blocks) that are typically non-unique and arbitrarily stored on-site; and 2) assemble these components into pre-designed modular structures. The algorithms use a single camera and a visual marker-based metrology to rapidly establish local reference frames and to detect staged building components. Based on the design of the structure being assembled, the algorithms automatically determine the assembly sequence. Furthermore, if a 3D camera is mounted on the manipulator, 3D point clouds can be readily captured and registered into a same reference frame through our marker-based metrology and the manipulator's internal encoders, either after construction to facilitate as-built Building Information Model (BIM) generation, or during construction to document details of the progress. Implemented using a 7-axis KUKA KR100 robotic manipulator, the presented robotic system has successfully assembled various structures and created as-built 3D point cloud models autonomously, demonstrating the designed algorithms' effectiveness in autonomous on-site construction robotics applications.

AB - Unlike robotics in the manufacturing industry, on-site construction robotics has to consider and address two unique challenges: 1) the rugged, evolving, and unstructured environment of typical work sites; and 2) the reversed spatial relationship between the product and the manipulator, i.e., the manipulator has to travel to and localize itself at the work face, rather than a partially complete product arriving at an anchored manipulator. The presented research designed and implemented algorithms that address these challenges and enable autonomous robotic assembly of freeform modular structures on construction sites. Building on the authors' previous work in computer-vision-based pose estimation, the designed algorithms enable a mobile robotic manipulator to: 1) autonomously identify and grasp prismatic building components (e.g., bricks, blocks) that are typically non-unique and arbitrarily stored on-site; and 2) assemble these components into pre-designed modular structures. The algorithms use a single camera and a visual marker-based metrology to rapidly establish local reference frames and to detect staged building components. Based on the design of the structure being assembled, the algorithms automatically determine the assembly sequence. Furthermore, if a 3D camera is mounted on the manipulator, 3D point clouds can be readily captured and registered into a same reference frame through our marker-based metrology and the manipulator's internal encoders, either after construction to facilitate as-built Building Information Model (BIM) generation, or during construction to document details of the progress. Implemented using a 7-axis KUKA KR100 robotic manipulator, the presented robotic system has successfully assembled various structures and created as-built 3D point cloud models autonomously, demonstrating the designed algorithms' effectiveness in autonomous on-site construction robotics applications.

KW - As-built 3D modeling

KW - Autonomous assembly

KW - On-site construction robotics

KW - Pose estimation

UR - http://www.scopus.com/inward/record.url?scp=84944441544&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944441544&partnerID=8YFLogxK

U2 - 10.1016/j.autcon.2015.06.002

DO - 10.1016/j.autcon.2015.06.002

M3 - Article

AN - SCOPUS:84944441544

VL - 59

SP - 128

EP - 138

JO - Automation in Construction

JF - Automation in Construction

SN - 0926-5805

M1 - 1920

ER -