Addressing the big-earth-data variety challenge with the hierarchical triangular mesh

Michael L. Rilee; Kwo Sen Kuo; Thomas Clune; Amidu Oloso; Paul G. Brown; Hongfeng Yu

doi:10.1109/BigData.2016.7840700

Addressing the big-earth-data variety challenge with the hierarchical triangular mesh

Michael L. Rilee, Kwo Sen Kuo, Thomas Clune, Amidu Oloso, Paul G. Brown, Hongfeng Yu

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

12 Scopus citations

Abstract

We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.

Original language	English (US)
Title of host publication	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
Editors	Ronay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1006-1011
Number of pages	6
ISBN (Electronic)	9781467390040
DOIs	https://doi.org/10.1109/BigData.2016.7840700
State	Published - 2016
Event	4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States Duration: Dec 5 2016 → Dec 8 2016

Publication series

Name	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Other

Other	4th IEEE International Conference on Big Data, Big Data 2016
Country/Territory	United States
City	Washington
Period	12/5/16 → 12/8/16

Keywords

DAAC
GIS
HTM
SciDB
array database
data analysis
data fusion
geographic metadata
indexing
load balancing
remote sensing
shared nothing architecture
variety

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Hardware and Architecture

Access to Document

10.1109/BigData.2016.7840700

Cite this

Rilee, M. L., Kuo, K. S., Clune, T., Oloso, A., Brown, P. G., & Yu, H. (2016). Addressing the big-earth-data variety challenge with the hierarchical triangular mesh. In R. Ak, G. Karypis, Y. Xia, X. T. Hu, P. S. Yu, J. Joshi, L. Ungar, L. Liu, A.-H. Sato, T. Suzumura, S. Rachuri, R. Govindaraju, & W. Xu (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 1006-1011). Article 7840700 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840700

Addressing the big-earth-data variety challenge with the hierarchical triangular mesh. / Rilee, Michael L.; Kuo, Kwo Sen; Clune, Thomas et al.
Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / Ronay Ak; George Karypis; Yinglong Xia; Xiaohua Tony Hu; Philip S. Yu; James Joshi; Lyle Ungar; Ling Liu; Aki-Hiro Sato; Toyotaro Suzumura; Sudarsan Rachuri; Rama Govindaraju; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1006-1011 7840700 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Rilee, ML, Kuo, KS, Clune, T, Oloso, A, Brown, PG & Yu, H 2016, Addressing the big-earth-data variety challenge with the hierarchical triangular mesh. in R Ak, G Karypis, Y Xia, XT Hu, PS Yu, J Joshi, L Ungar, L Liu, A-H Sato, T Suzumura, S Rachuri, R Govindaraju & W Xu (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840700, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, Institute of Electrical and Electronics Engineers Inc., pp. 1006-1011, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 12/5/16. https://doi.org/10.1109/BigData.2016.7840700

Rilee ML, Kuo KS, Clune T, Oloso A, Brown PG, Yu H. Addressing the big-earth-data variety challenge with the hierarchical triangular mesh. In Ak R, Karypis G, Xia Y, Hu XT, Yu PS, Joshi J, Ungar L, Liu L, Sato AH, Suzumura T, Rachuri S, Govindaraju R, Xu W, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1006-1011. 7840700. (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). doi: 10.1109/BigData.2016.7840700

Rilee, Michael L. ; Kuo, Kwo Sen ; Clune, Thomas et al. / Addressing the big-earth-data variety challenge with the hierarchical triangular mesh. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / Ronay Ak ; George Karypis ; Yinglong Xia ; Xiaohua Tony Hu ; Philip S. Yu ; James Joshi ; Lyle Ungar ; Ling Liu ; Aki-Hiro Sato ; Toyotaro Suzumura ; Sudarsan Rachuri ; Rama Govindaraju ; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1006-1011 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

@inproceedings{ef23e537887e44a78511b9ae95673994,

title = "Addressing the big-earth-data variety challenge with the hierarchical triangular mesh",

abstract = "We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.",

keywords = "DAAC, GIS, HTM, SciDB, array database, data analysis, data fusion, geographic metadata, indexing, load balancing, remote sensing, shared nothing architecture, variety",

author = "Rilee, {Michael L.} and Kuo, {Kwo Sen} and Thomas Clune and Amidu Oloso and Brown, {Paul G.} and Hongfeng Yu",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 4th IEEE International Conference on Big Data, Big Data 2016 ; Conference date: 05-12-2016 Through 08-12-2016",

year = "2016",

doi = "10.1109/BigData.2016.7840700",

language = "English (US)",

series = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1006--1011",

editor = "Ronay Ak and George Karypis and Yinglong Xia and Hu, {Xiaohua Tony} and Yu, {Philip S.} and James Joshi and Lyle Ungar and Ling Liu and Aki-Hiro Sato and Toyotaro Suzumura and Sudarsan Rachuri and Rama Govindaraju and Weijia Xu",

booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

}

TY - GEN

T1 - Addressing the big-earth-data variety challenge with the hierarchical triangular mesh

AU - Rilee, Michael L.

AU - Kuo, Kwo Sen

AU - Clune, Thomas

AU - Oloso, Amidu

AU - Brown, Paul G.

AU - Yu, Hongfeng

PY - 2016

Y1 - 2016

N2 - We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.

AB - We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.

KW - DAAC

KW - GIS

KW - HTM

KW - SciDB

KW - array database

KW - data analysis

KW - data fusion

KW - geographic metadata

KW - indexing

KW - load balancing

KW - remote sensing

KW - shared nothing architecture

KW - variety

UR - http://www.scopus.com/inward/record.url?scp=85015169222&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015169222&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840700

DO - 10.1109/BigData.2016.7840700

M3 - Conference contribution

AN - SCOPUS:85015169222

T3 - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

SP - 1006

EP - 1011

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Ak, Ronay

A2 - Karypis, George

A2 - Xia, Yinglong

A2 - Hu, Xiaohua Tony

A2 - Yu, Philip S.

A2 - Joshi, James

A2 - Ungar, Lyle

A2 - Liu, Ling

A2 - Sato, Aki-Hiro

A2 - Suzumura, Toyotaro

A2 - Rachuri, Sudarsan

A2 - Govindaraju, Rama

A2 - Xu, Weijia

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th IEEE International Conference on Big Data, Big Data 2016

Y2 - 5 December 2016 through 8 December 2016

ER -

Addressing the big-earth-data variety challenge with the hierarchical triangular mesh

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this