In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Qian Sun, Melissa Romanus, Tong Jin, Hongfeng Yu, Peer Timo Bremer, Steve Petruzza, Scott Klasky, Manish Parashar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

Original languageEnglish (US)
Title of host publicationProceedings of ESPM2 2016
Subtitle of host publication2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2-9
Number of pages8
ISBN (Electronic)9781509038589
DOIs
StatePublished - Jan 24 2017
Event2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016 - Salt Lake City, United States
Duration: Nov 18 2016 → …

Publication series

NameProceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

Other

Other2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016
CountryUnited States
CitySalt Lake City
Period11/18/16 → …

Keywords

  • Couplings
  • Data storage systems
  • Runtime

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software

Fingerprint Dive into the research topics of 'In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows'. Together they form a unique fingerprint.

Cite this