TY - GEN
T1 - Towards load balancing support for I/O-intensive parallel jobs in a cluster of workstations
AU - Qin, Xiao
AU - Jiang, Hong
AU - Zhu, Yifeng
AU - Swanson, David R.
N1 - Publisher Copyright:
© 2003 IEEE.
PY - 2003
Y1 - 2003
N2 - While previous CPU- or memory-centric load balancing schemes are capable of achieving the effective usage of global CPU and memory resources in a cluster system, the cluster exhibits significant performance drop under I/O-intensive workload conditions due to the imbalance of I/O load. To tackle this problem, we have developed two simple yet effective I/O-aware load-balancing schemes, which make it possible to balance I/O load by assigning I/O intensive sequential and parallel jobs to nodes with light I/O loads. Moreover, the proposed schemes judiciously take into account both CPU and memory load sharing in the cluster, thereby maintaining a high performance for a wide spectrum of workload. Using a set of real I/O-intensive parallel applications in addition to synthetic parallel jobs, we show that the proposed schemes consistently outperform the existing non-I/O-aware load-balancing schemes for a diverse set of workload conditions. Importantly, the performance improvement becomes much more pronounced when the applications are I/O-intensive.
AB - While previous CPU- or memory-centric load balancing schemes are capable of achieving the effective usage of global CPU and memory resources in a cluster system, the cluster exhibits significant performance drop under I/O-intensive workload conditions due to the imbalance of I/O load. To tackle this problem, we have developed two simple yet effective I/O-aware load-balancing schemes, which make it possible to balance I/O load by assigning I/O intensive sequential and parallel jobs to nodes with light I/O loads. Moreover, the proposed schemes judiciously take into account both CPU and memory load sharing in the cluster, thereby maintaining a high performance for a wide spectrum of workload. Using a set of real I/O-intensive parallel applications in addition to synthetic parallel jobs, we show that the proposed schemes consistently outperform the existing non-I/O-aware load-balancing schemes for a diverse set of workload conditions. Importantly, the performance improvement becomes much more pronounced when the applications are I/O-intensive.
UR - http://www.scopus.com/inward/record.url?scp=84944893513&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84944893513&partnerID=8YFLogxK
U2 - 10.1109/CLUSTR.2003.1253305
DO - 10.1109/CLUSTR.2003.1253305
M3 - Conference contribution
AN - SCOPUS:84944893513
T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC
SP - 100
EP - 107
BT - Proceedings - IEEE International Conference on Cluster Computing, CLUSTER 2003
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - IEEE International Conference on Cluster Computing, CLUSTER 2003
Y2 - 1 December 2003 through 4 December 2003
ER -