Adaptive Deep Learning based Time-Varying Volume Compression

Yu Pan, Feiyu Zhu, Tian Gao, Hongfeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


Nowadays, floating-point temporal-spatial datasets are routinely generated from scientific observational apparatuses or computer simulations at an unprecedented pace. The sheer amount of these large volumetric datasets on the order of terabytes or petabytes consume massive resources in terms of bandwidth, storage and computational power. On the other hand, scientists, equipped with low-end post-analysis machines, often find it impossible to visualize and analyze these massive datasets with such limited resources in hand, not to mention their ultimate goal of real time analysis and visualization. To solve this discrepancy, a compact data representation has to be generated and a trade-off between resource consumption and analytical precision has to be found. There are many existing volumetric representation generating methods, almost all of which adopts some kind of hand-engineered heuristics to extract the effective portion of the datasets. However, the trade-off between resource consumption and analytical quality could not be well established due to the introduction of hand-engineered heuristics. In this paper, we present a deep learning based method that can adaptively capture the inherently complicated dynamics of temporal-spatial volumetric datasets without introducing any hand engineered features. We train an autoencoder based neural network with quantization and adaptation. Compared with existing methods, our method could learn data representation at a much lower compressed/uncompressed rate while preserving the details of original datasets. Also, our method could adapt with different data distribution and conduct compression and decompression in real time. Through extensive experiments, we show the effectiveness and efficiency of our approach over existing methods.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019
EditorsChaitanya Baru, Jun Huan, Latifur Khan, Xiaohua Tony Hu, Ronay Ak, Yuanyuan Tian, Roger Barga, Carlo Zaniolo, Kisung Lee, Yanfang Fanny Ye
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages8
ISBN (Electronic)9781728108582
StatePublished - Dec 2019
Externally publishedYes
Event2019 IEEE International Conference on Big Data, Big Data 2019 - Los Angeles, United States
Duration: Dec 9 2019Dec 12 2019

Publication series

NameProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019


Conference2019 IEEE International Conference on Big Data, Big Data 2019
Country/TerritoryUnited States
CityLos Angeles


  • autoencoder
  • deep learning
  • scientific data
  • volume compression

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management


Dive into the research topics of 'Adaptive Deep Learning based Time-Varying Volume Compression'. Together they form a unique fingerprint.

Cite this