A dynamic load balancing scheme for I/O-intensive applications in distributed systems

Xiao Qin, Hong Jiang, Yifeng Zhu, D. R. Swanson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations

Abstract

In this paper, a new I/O-aware load-balancing scheme is presented to improve overall performance of a distributed system with a general and practical workload including I/O activities. The proposed scheme dynamically detects I/O load imbalance on nodes of a distributed system and determines whether to migrate the I/O requests of some jobs from overloaded nodes to other less- or under-loaded nodes, depending on data migration cost and remote I/O access overhead. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in distributed systems, thereby maintaining the same level of performance as the existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation study show that, compared with the existing schemes that only consider CPU and memory, the proposed scheme reduces the mean slowdown by up to 54.5% (with an average of 39.9%). On the other hand, when compared to the existing approaches that only consider I/O, the proposed scheme reduces the mean slowdown by up to 57.2% (with an average of 31.6%). More importantly, the new scheme improves over a very recent algorithm found in the literature that considers all the three resources by up to 49.6% (with an average of up to 41.9%).

Original languageEnglish (US)
Title of host publicationProceedings - 2003 International Conference on Parallel Processing Workshops, ICPPW 2003
EditorsChua-Huang Huang, J. Ramanujam
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages79-86
Number of pages8
ISBN (Electronic)0769520189
DOIs
StatePublished - Jan 1 2003
Event2003 International Conference on Parallel Processing Workshops, ICPPW 2003 - Kaohsiung, Taiwan, Province of China
Duration: Oct 6 2003Oct 9 2003

Publication series

NameProceedings of the International Conference on Parallel Processing Workshops
Volume2003-January
ISSN (Print)1530-2016

Other

Other2003 International Conference on Parallel Processing Workshops, ICPPW 2003
CountryTaiwan, Province of China
CityKaohsiung
Period10/6/0310/9/03

Keywords

  • Application software
  • Clustering algorithms
  • Computational modeling
  • Computer architecture
  • Computer science
  • Costs
  • Load management
  • Remote sensing
  • System performance
  • Workstations

ASJC Scopus subject areas

  • Software
  • Mathematics(all)
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'A dynamic load balancing scheme for I/O-intensive applications in distributed systems'. Together they form a unique fingerprint.

  • Cite this

    Qin, X., Jiang, H., Zhu, Y., & Swanson, D. R. (2003). A dynamic load balancing scheme for I/O-intensive applications in distributed systems. In C-H. Huang, & J. Ramanujam (Eds.), Proceedings - 2003 International Conference on Parallel Processing Workshops, ICPPW 2003 (pp. 79-86). [1240357] (Proceedings of the International Conference on Parallel Processing Workshops; Vol. 2003-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICPPW.2003.1240357