Reinforcement Learning Based Optimization of Query Execution Plans in Distributed Databases

Abstract

Troublesome workloads, data heterogeneity, and shifting resource conditions make efficient query execution highly difficult to achieve in distributed database systems. Traditional optimizers will almost always rely on handcrafted methods or static cost models to achieve the desired results, resulting in adaptative failures along the way and serving at best subpar query execution plans (QEPs). This paper presents a new architecture meant to optimize QEPs by utilizing deep policy reinforcement learning (RL) for dynamically shifting execution strategy adaptations over distributed nodes. The proposed model considers and structures the optimization problem as a Markov Decision Process (MDP) with states available in the form of system and query profiles, actions available being the choices of QEPs, and the rewards acting as a mere performance measurement for execution. We analyze this approach with different combinations of queries and nodes through benchmark datasets and simulated environments. The objective of this evaluation is to test the model’s performance in regards to differing query kinds and node configurations. The experiments indicate remarkable advances in system throughput and execution time while achieving strong generalization to unfamiliar queries. These results support the hypothesized ability of query processing in future distributed databases to not have suggestion mechanisms reliant on rules or costs, unlike their predecessors, and instead implement optimizers that utilize RL.