Towards load balancing support for I/O-intensive parallel jobs in a cluster of workstations

Xiao Qin, Hong Jiang, Yifeng Zhu, David R. Swanson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Scopus citations

Abstract

While previous CPU- or memory-centric load balancing schemes are capable of achieving the effective usage of global CPU and memory resources in a cluster system, the cluster exhibits significant performance drop under I/O-intensive workload conditions due to the imbalance of I/O load. To tackle this problem, we have developed two simple yet effective I/O-aware load-balancing schemes, which make it possible to balance I/O load by assigning I/O intensive sequential and parallel jobs to nodes with light I/O loads. Moreover, the proposed schemes judiciously take into account both CPU and memory load sharing in the cluster, thereby maintaining a high performance for a wide spectrum of workload. Using a set of real I/O-intensive parallel applications in addition to synthetic parallel jobs, we show that the proposed schemes consistently outperform the existing non-I/O-aware load-balancing schemes for a diverse set of workload conditions. Importantly, the performance improvement becomes much more pronounced when the applications are I/O-intensive.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Conference on Cluster Computing, CLUSTER 2003
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages100-107
Number of pages8
ISBN (Electronic)0769520669
DOIs
StatePublished - 2003
EventIEEE International Conference on Cluster Computing, CLUSTER 2003 - Hong Kong, China
Duration: Dec 1 2003Dec 4 2003

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Volume2003-January
ISSN (Print)1552-5244

Other

OtherIEEE International Conference on Cluster Computing, CLUSTER 2003
CountryChina
CityHong Kong
Period12/1/0312/4/03

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Signal Processing

Fingerprint Dive into the research topics of 'Towards load balancing support for I/O-intensive parallel jobs in a cluster of workstations'. Together they form a unique fingerprint.

  • Cite this

    Qin, X., Jiang, H., Zhu, Y., & Swanson, D. R. (2003). Towards load balancing support for I/O-intensive parallel jobs in a cluster of workstations. In Proceedings - IEEE International Conference on Cluster Computing, CLUSTER 2003 (pp. 100-107). [1253305] (Proceedings - IEEE International Conference on Cluster Computing, ICCC; Vol. 2003-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CLUSTR.2003.1253305