Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system

X. Qin, H. Jiang, Y. Zhu, D. R. Swanson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.

Original languageEnglish (US)
Title of host publicationProceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003
EditorsLiria M. Sato, Edson T. Midorikawa, Philippe O. A. Navaux
PublisherIEEE Computer Society
Pages235-243
Number of pages9
ISBN (Electronic)0769520464
DOIs
StatePublished - 2003
Event15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003 - Sao Paulo, Brazil
Duration: Nov 10 2003Nov 12 2003

Publication series

NameProceedings - Symposium on Computer Architecture and High Performance Computing
Volume2003-January
ISSN (Print)1550-6533

Other

Other15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003
CountryBrazil
CitySao Paulo
Period11/10/0311/12/03

Keywords

  • Application software
  • Boosting
  • Bridges
  • Computational modeling
  • Computer science
  • Costs
  • Load management
  • Remote sensing
  • System performance
  • Throughput

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software

Fingerprint Dive into the research topics of 'Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system'. Together they form a unique fingerprint.

  • Cite this

    Qin, X., Jiang, H., Zhu, Y., & Swanson, D. R. (2003). Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. In L. M. Sato, E. T. Midorikawa, & P. O. A. Navaux (Eds.), Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003 (pp. 235-243). [1250343] (Proceedings - Symposium on Computer Architecture and High Performance Computing; Vol. 2003-January). IEEE Computer Society. https://doi.org/10.1109/CAHPC.2003.1250343