Comparison between evolutionary algorithms and Reinforcement Learning for workflow autoscaling on Cloud

Abstract

In recent years, many scientific experiments have been performed using scientific workflows. These technologies facilitate the  performance of computationally intensive experiments, which often require to be executed on public Clouds. Optimizing these applications becomes a challenging problem since resource virtualization demands scheduling needs to be satisfied, at the same time it has to deal with uncertainties during execution. For this reason, heuristic and metaheuristic solutions have been proposed to this problem. Indeed, Reinforcement Learning and evolutionary algorithms have been used to tackled this problem. This paper presents a Markovian Decision Problem that can be used to solve this problem using Reinforcement Learning. Additionally, a variation of this modeling is introduced to solve the same problem using multi-objective evolutionary algorithms. These two strategies are compared using 4 benchmark workflows using the simulator CloudSimPlus and virtual machines present on Amazon public clouds. Total monetary cost of the execution, total execution time (i.e. makespan), and the L2 norm of these two quantities are used for the comparative analysis.