Adaptive application-aware job scheduling optimization strategy in heterogeneous infrastructures

Abstract

As the rapid advancement and diversity in the computing systems, it is demanding to take the most robust scheduling algorithm that guarantee an optimized performance to execute manifold applications on largescale heterogeneous computing environments. This paperpresent an adaptive application-aware job scheduling optimization strategy for large-scale high throughput computing in heterogeneous infrastructures. The proposed scheduling optimization method is built on two main concepts. First, it provides application-aware job distribution weights through empirical data in large-scale heterogeneous infrastructures. Here we adopt the concept of weight, which represent the ratio of tasks that will be computed on each resource. The weights can vary in terms of application type and are optimized until the system get in steady status. Secondly, it offers an adaptive control phase that is invoked by the weight adjustment and resource scaling feature. The feedback data frommonitoring module is forwarded to the control phase in order to adjust weights and over-provisioning ratio, and result in enhancing overall balance between performances and utilization of system. The experimental evaluation with the four realistic workload patterns demonstrates that, when comparedto the core-based scheme which distributes tasks in proportions of each resource’s number of cores, the use of our optimization method can achieve 62% better average throughput, 43% shorter average queueing time, and 38% better average utilization of the entire resources in diverseinfrastructure environments by harnessing our adaptive module.