Multi-Workflow Concurrent Scheduling in a Heterogeneous Computing Cluster

Abstract

Scheduling scientific workflow in a distributed computing resource is a challenging job. It involves heterogeneous resource allocation to concurrent tasks in order to achieve the desired scheduling goal. In this paper, we are presenting a multi-workflow Earliest Cycle Time algorithm, mECT. The objective of the algorithm is to reduce schedule length while ensuring no deadlock situation occurs due to dependency violation. The algorithm is suitable for multi workflow applications that deal with large data sets and run in a cyclic order. We have carried out extensive simulations to compare the proposed algorithm with well known existing algorithms. We have also tested our algorithm for a low resolution (N48) weather Unified Model (UM-10.2) in a simulated environment developed by the Met Office, United Kingdom. Results show that mECT performs better in terms of shorter makespan and lesser deadlocks.