Lessons learned with laser scanning point cloud management in Hadoop HBase

Anh Vu Vo, Nikita Konda, Neel Chauhan, Harith Aljumaily, Debra Laefer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While big data technologies are growing rapidly and benefit a wide range of science and engineering domains, many barriers remain for the remote sensing community to fully exploit the benefits provided by these powerful and rapidly developing technologies. To overcome existing barriers, this paper presents the in-depth experience gained when adopting a distributed computing framework – Hadoop HBase – for storage, indexing, and integration of large scale, high resolution laser scanning point cloud data. Four data models were conceptualized, implemented, and rigorously investigated to explore the advantageous features of distributed, key-value database systems. In addition, the comparison of the four models facilitated the reassessment of several well-known point cloud management techniques founded in traditional computing environments in the new context of a distributed, key-value database. The four models were derived from two row-key designs and two columns structures, thereby demonstrating various considerations during the development of a data solution for high-resolution, city-scale aerial laser scan for a portion of Dublin, Ireland. This paper presents lessons learned from the data model design and its implementation for spatial data management in a distributed computing framework. The study is a step towards full exploitation of powerful emerging computing assets for dense spatio-temporal data.

Original languageEnglish (US)
Title of host publicationAdvanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings
PublisherSpringer-Verlag
Pages231-253
Number of pages23
ISBN (Print)9783319916347
DOIs
StatePublished - Jan 1 2018
Event25th Workshop of the European Group for Intelligent Computing in Engineering, EG-ICE 2018 - Lausanne, Switzerland
Duration: Jun 10 2018Jun 13 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10863 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th Workshop of the European Group for Intelligent Computing in Engineering, EG-ICE 2018
CountrySwitzerland
CityLausanne
Period6/10/186/13/18

Fingerprint

Laser Scanning
Point Cloud
Distributed computer systems
Data structures
Distributed Computing
Scanning
Data Model
Lasers
High Resolution
Information management
Remote sensing
Spatio-temporal Data
Computing
Spatial Data
Database Systems
Antennas
Data Management
Indexing
Remote Sensing
Exploitation

Keywords

  • Big data
  • Distributed database
  • Hadoop
  • HBase
  • LiDAR
  • Point cloud
  • Spatial data management

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Vo, A. V., Konda, N., Chauhan, N., Aljumaily, H., & Laefer, D. (2018). Lessons learned with laser scanning point cloud management in Hadoop HBase. In Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings (pp. 231-253). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10863 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-91635-4_13

Lessons learned with laser scanning point cloud management in Hadoop HBase. / Vo, Anh Vu; Konda, Nikita; Chauhan, Neel; Aljumaily, Harith; Laefer, Debra.

Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings. Springer-Verlag, 2018. p. 231-253 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10863 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Vo, AV, Konda, N, Chauhan, N, Aljumaily, H & Laefer, D 2018, Lessons learned with laser scanning point cloud management in Hadoop HBase. in Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10863 LNCS, Springer-Verlag, pp. 231-253, 25th Workshop of the European Group for Intelligent Computing in Engineering, EG-ICE 2018, Lausanne, Switzerland, 6/10/18. https://doi.org/10.1007/978-3-319-91635-4_13
Vo AV, Konda N, Chauhan N, Aljumaily H, Laefer D. Lessons learned with laser scanning point cloud management in Hadoop HBase. In Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings. Springer-Verlag. 2018. p. 231-253. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-91635-4_13
Vo, Anh Vu ; Konda, Nikita ; Chauhan, Neel ; Aljumaily, Harith ; Laefer, Debra. / Lessons learned with laser scanning point cloud management in Hadoop HBase. Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings. Springer-Verlag, 2018. pp. 231-253 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{069602beadf74edeb638514832bb8de0,
title = "Lessons learned with laser scanning point cloud management in Hadoop HBase",
abstract = "While big data technologies are growing rapidly and benefit a wide range of science and engineering domains, many barriers remain for the remote sensing community to fully exploit the benefits provided by these powerful and rapidly developing technologies. To overcome existing barriers, this paper presents the in-depth experience gained when adopting a distributed computing framework – Hadoop HBase – for storage, indexing, and integration of large scale, high resolution laser scanning point cloud data. Four data models were conceptualized, implemented, and rigorously investigated to explore the advantageous features of distributed, key-value database systems. In addition, the comparison of the four models facilitated the reassessment of several well-known point cloud management techniques founded in traditional computing environments in the new context of a distributed, key-value database. The four models were derived from two row-key designs and two columns structures, thereby demonstrating various considerations during the development of a data solution for high-resolution, city-scale aerial laser scan for a portion of Dublin, Ireland. This paper presents lessons learned from the data model design and its implementation for spatial data management in a distributed computing framework. The study is a step towards full exploitation of powerful emerging computing assets for dense spatio-temporal data.",
keywords = "Big data, Distributed database, Hadoop, HBase, LiDAR, Point cloud, Spatial data management",
author = "Vo, {Anh Vu} and Nikita Konda and Neel Chauhan and Harith Aljumaily and Debra Laefer",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-319-91635-4_13",
language = "English (US)",
isbn = "9783319916347",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "231--253",
booktitle = "Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings",

}

TY - GEN

T1 - Lessons learned with laser scanning point cloud management in Hadoop HBase

AU - Vo, Anh Vu

AU - Konda, Nikita

AU - Chauhan, Neel

AU - Aljumaily, Harith

AU - Laefer, Debra

PY - 2018/1/1

Y1 - 2018/1/1

N2 - While big data technologies are growing rapidly and benefit a wide range of science and engineering domains, many barriers remain for the remote sensing community to fully exploit the benefits provided by these powerful and rapidly developing technologies. To overcome existing barriers, this paper presents the in-depth experience gained when adopting a distributed computing framework – Hadoop HBase – for storage, indexing, and integration of large scale, high resolution laser scanning point cloud data. Four data models were conceptualized, implemented, and rigorously investigated to explore the advantageous features of distributed, key-value database systems. In addition, the comparison of the four models facilitated the reassessment of several well-known point cloud management techniques founded in traditional computing environments in the new context of a distributed, key-value database. The four models were derived from two row-key designs and two columns structures, thereby demonstrating various considerations during the development of a data solution for high-resolution, city-scale aerial laser scan for a portion of Dublin, Ireland. This paper presents lessons learned from the data model design and its implementation for spatial data management in a distributed computing framework. The study is a step towards full exploitation of powerful emerging computing assets for dense spatio-temporal data.

AB - While big data technologies are growing rapidly and benefit a wide range of science and engineering domains, many barriers remain for the remote sensing community to fully exploit the benefits provided by these powerful and rapidly developing technologies. To overcome existing barriers, this paper presents the in-depth experience gained when adopting a distributed computing framework – Hadoop HBase – for storage, indexing, and integration of large scale, high resolution laser scanning point cloud data. Four data models were conceptualized, implemented, and rigorously investigated to explore the advantageous features of distributed, key-value database systems. In addition, the comparison of the four models facilitated the reassessment of several well-known point cloud management techniques founded in traditional computing environments in the new context of a distributed, key-value database. The four models were derived from two row-key designs and two columns structures, thereby demonstrating various considerations during the development of a data solution for high-resolution, city-scale aerial laser scan for a portion of Dublin, Ireland. This paper presents lessons learned from the data model design and its implementation for spatial data management in a distributed computing framework. The study is a step towards full exploitation of powerful emerging computing assets for dense spatio-temporal data.

KW - Big data

KW - Distributed database

KW - Hadoop

KW - HBase

KW - LiDAR

KW - Point cloud

KW - Spatial data management

UR - http://www.scopus.com/inward/record.url?scp=85049075524&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049075524&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-91635-4_13

DO - 10.1007/978-3-319-91635-4_13

M3 - Conference contribution

AN - SCOPUS:85049075524

SN - 9783319916347

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 231

EP - 253

BT - Advanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings

PB - Springer-Verlag

ER -