A Hybrid Deep Q-Learning And Optimal Queuing Framework For Adaptive Cloud Task Scheduling

Abstract

The rapid expansion of cloud and edge computing infrastructures has transformed the way computational services are delivered, enabling elastic, on-demand, and distributed processing for data-intensive and latency-sensitive applications. However, this transformation has also generated unprecedented challenges in dynamic task scheduling, resource allocation, and quality-of-service assurance, particularly under volatile workloads and heterogeneous system conditions. Classical deterministic and heuristic scheduling techniques, originally developed for static or quasi-static computing environments, have increasingly demonstrated structural limitations when confronted with the stochasticity, scale, and interdependence that characterize modern cloud ecosystems. In response to these challenges, deep reinforcement learning has emerged as a powerful paradigm for adaptive and data-driven decision making in complex computational environments, offering the potential to learn optimal scheduling strategies directly from system interactions rather than from predefined rules (Cheng et al., 2018; Ding et al., 2020; Kanikanti et al., 2025).

This study develops and critically evaluates an integrated theoretical and methodological framework that combines deep Q-learning with optimal queuing theory to model, analyze, and improve dynamic task scheduling in cloud computing environments. Building on the insights of Kanikanti et al. (2025), who demonstrated that deep Q-learning driven scheduling can be significantly enhanced through the incorporation of optimal queuing principles, the present research advances the conceptual foundations of intelligent scheduling by embedding learning-based control within a structured stochastic service system. This synthesis enables the scheduling agent to reason not only about immediate rewards, such as execution time or energy consumption, but also about long-term queue stability, waiting time distributions, and system-wide congestion effects. Through this integration, the framework seeks to overcome the myopic tendencies of conventional reinforcement learning schedulers while avoiding the rigidity of purely analytical queuing models.

The article situates this hybrid approach within the broader scholarly discourse on reinforcement learning, cloud resource management, and intelligent systems design. Drawing on diverse strands of literature including deep reinforcement learning for cloud and edge computing (Choppara and Mangalampalli, 2025; Anand and Karthikeyan, 2025; Wang et al., 2021), multi-agent and modular learning systems (Pan and Wu, 2025; Wang et al., 2025), and anomaly-aware and risk-sensitive modeling (Kardani-Moghaddam et al., 2021; Lian et al., 2025), the study elaborates a comprehensive perspective on how learning-driven schedulers can be made more robust, interpretable, and scalable. The methodological design emphasizes descriptive and analytical reasoning rather than numerical simulation, focusing on how theoretical constructs and algorithmic mechanisms interact to shape emergent system behavior.

The results, interpreted through the lens of the cited literature, suggest that deep Q-learning integrated with optimal queuing frameworks offers a fundamentally different mode of scheduling intelligence. Rather than merely reacting to instantaneous system states, the scheduler internalizes long-term structural knowledge about service dynamics, enabling more stable, fair, and efficient task allocation under fluctuating workloads (Kanikanti et al., 2025; Shang et al., 2022). This leads to improved conceptual performance in terms of latency, throughput, and resource utilization when compared to both rule-based and purely learning-based approaches.

The discussion extends these findings by exploring theoretical implications for the future of autonomous cloud management, including the role of modular adapters, knowledge injection, and large-scale model composition in reinforcement learning systems (Zheng et al., 2025; Wang et al., 2025). It also critically examines potential limitations, such as model complexity, convergence stability, and interpretability, and proposes directions for future research that integrate semantic knowledge graphs, anomaly detection, and financial risk modeling into cloud scheduling architectures (Yan et al., 2024; Chiang et al., 2025; Xu et al., 2025). By presenting an in-depth, citation-grounded, and theoretically rich exploration of deep reinforcement learning driven scheduling, this article contributes to a more holistic understanding of how intelligent control can be realized in next-generation cloud and edge computing systems.