Hierarchical Deep RL for Sustainable API Throughput and Latency Optimization in Advertising Clouds

Aisha Tan; Kumar Rajesh

doi:10.54097/5a4fg660

Authors

Aisha Tan
Kumar Rajesh

DOI:

https://doi.org/10.54097/5a4fg660

Keywords:

Hierarchical Deep Reinforcement Learning, API Optimization, Advertising Clouds, Sustainable Computing, Deep Q-Networks, Advantage Actor-Critic, Throughput Optimization, Latency Minimization

Abstract

Advertising cloud platforms face escalating challenges in Application Programming Interface (API) performance optimization due to diverse client request patterns, fluctuating workloads, and increasing sustainability requirements. Traditional API management approaches struggle to balance throughput maximization with latency minimization while considering energy efficiency and carbon footprint reduction. The heterogeneous nature of advertising APIs, including bidding interfaces, content delivery services, and analytics endpoints, requires sophisticated optimization strategies that adapt to varying performance requirements and resource constraints. This study proposes a Hierarchical Deep Reinforcement Learning (HDRL) framework for sustainable API throughput and latency optimization in advertising cloud environments. The framework employs a multi-level architecture where global orchestrators manage cross-API resource allocation while local optimizers focus on individual API performance tuning. Deep Q-Networks (DQNs) and Advantage Actor-Critic (A2C) algorithms enable adaptive optimization policies that simultaneously maximize API throughput, minimize response latency, and reduce energy consumption across distributed cloud infrastructure. Experimental evaluation using production advertising cloud workloads demonstrates that the proposed framework achieves 44% improvement in API throughput while reducing average response latency by 39% compared to traditional optimization methods. The sustainability-focused approach decreases energy consumption by 35% and carbon emissions by 42%, while maintaining Service Level Agreement (SLA) compliance rates above 96% across all API categories.

Downloads

Download data is not yet available.

References

[1] Ji, E., Wang, Y., Xing, S., & Jin, J. (2025). Hierarchical Reinforcement Learning for Energy-Efficient API Traffic Optimization in Large-Scale Advertising Systems. IEEE Access

[2] Vashishth, T. K., Sharma, V., Kumar, B., & Sharma, K. K. (2024). Cloud-Based Data Management for Behavior Analytics in Business and Finance Sectors. In Data-Driven Modelling and Predictive Analytics in Business and Finance (pp. 133-155). Auerbach Publications.

[3] Ahuja, A. (2024). Sustainable and ESG-Driven Multi-Cloud Optimization in Large Enterprises: Balancing Cost, Performance, and Flexibility. Well Testing Journal, 33(S2), 405-434.

[4] Rodrigues, D. N., Rosas, F. S., & Grácio, M. C. C. (2025). Dynamic Resource Allocation in Serverless ETL: AI-Driven Scaling and Cost Optimization Models.

[5] Kuranage, M. P. J. (2024). AI-driven Zero-Touch solutions for resource management in cloud-native 5G networks (Doctoral dissertation, Ecole nationale supérieure Mines-Télécom Atlantique).

[6] Yadav, P. S. (2024). Optimizing Serverless Architectures for Ultra-Low Latency in Financial Applications. European Journal of Advances in Engineering and Technology, 11(3), 146-157.

[7] Wilson, A., & Anwar, M. R. (2024). The future of adaptive machine learning algorithms in high-dimensional data processing. International Transactions on Artificial Intelligence, 3(1), 97-107.

[8] Tanasescu, A., Bucur, C., Andrei, J. V., Tudorica, B., Paraschiv, D., & Dusmanescu, D. (2025). A Deep Q-Networks Model for Optimising Decision-Making Process in the Context of Energy Transition Modelling. Environmental Research Communications.

[9] Jajan, K. I. K., & Zeebaree, S. R. (2024). Optimizing performance in distributed cloud architectures: A review of optimization techniques and tools. The Indonesian Journal of Computer Science, 13(2).

[10] Singh, N., Hamid, Y., Juneja, S., Srivastava, G., Dhiman, G., Gadekallu, T. R., & Shah, M. A. (2023). Load balancing and service discovery using Docker Swarm for microservice based big data applications. Journal of Cloud Computing, 12(1), 4.

[11] Subramanian, H., & Raj, P. (2019). Hands-On RESTful API Design Patterns and Best Practices: Design, develop, and deploy highly adaptable, scalable, and secure RESTful web APIs. Packt Publishing Ltd.

[12] Chinesta Llobregat, S. (2024). Design of a Data Analysis Platform as a Multitenant Service in the Cloud: An Approach towards Scalability and Adaptability.

[13] Rahimi, I., Gandomi, A. H., Chen, F., & Mezura-Montes, E. (2023). A review on constraint handling techniques for population-based algorithms: from single-objective to multi-objective optimization. Archives of Computational Methods in Engineering, 30(3), 2181-2209.

[14] Ogunwole, O., Onukwulu, E. C., Sam-Bulya, N. J., Joel, M. O., & Achumie, G. O. (2022). Optimizing automated pipelines for realtime data processing in digital media and e-commerce. International Journal of Multidisciplinary Research and Growth Evaluation, 3(1), 112-120.

[15] Mohammed, F., Olayah, F., Ali, A., & Gazem, N. A. (2020). The effect of cloud computing adoption on the sustainability of e-government services: A review. International Journal of Advanced Science and Technology, 29(5), 2636-2642.

[16] Beena, B. M., Prashanth, C. S. R., Manideep, T. S. S., Saragadam, S., & Karthik, G. (2025). A Green Cloud-Based Framework for Energy-Efficient Task Scheduling Using Carbon Intensity Data for Heterogeneous Cloud Servers. IEEE Access.

[17] Nagarajan, S., Rani, P. S., Vinmathi, M. S., Subba Reddy, V., Saleth, A. L. M., & Abdus Subhahan, D. (2025). Multi agent deep reinforcement learning for resource allocation in container-based clouds environments. Expert Systems, 42(1), e13362.

[18] Jiang, B., Wu, B., Cao, J., & Tan, Y. (2025). Interpretable Fair Value Hierarchy Classification via Hybrid Transformer-GNN Architecture. IEEE Access.

[19] Sætre, S. L. (2023). Deep Reinforcement Learning Based Parameter Optimisation for Installation Analysis of Marine Cables (Master's thesis, NTNU).

[20] Cao, W., & Mai, N. (2025). Predictive Analytics for Student Success: AI-Driven Early Warning Systems and Intervention Strategies for Educational Risk Management. Educational Research and Human Development, 2(2), Pp 36-48.

[21] Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022). Hierarchical reinforcement learning: A survey and open research challenges. Machine Learning and Knowledge Extraction, 4(1), 172-221.

[22] Cao, W., Mai, N., & Liu, W. (2025). Adaptive Knowledge Assessment via Symmetric Hierarchical Bayesian Neural Networks with Graph Symmetry-Aware Concept Dependencies. Symmetry.

[23] Suleiman, N., & Murtaza, Y. (2024). Scaling microservices for enterprise applications: Comprehensive strategies for achieving high availability, performance optimization, resilience, and seamless integration in large-scale distributed systems and complex cloud environments. Applied Research in Artificial Intelligence and Cloud Computing, 7(6), 46-82.

[24] Xing, S., Wang, Y., & Liu, W. (2025). Self-Adapting CPU Scheduling for Mixed Database Workloads via Hierarchical Deep Reinforcement Learning. Symmetry, 17(7), 1109.

[25] Hosseinzadeh, M., Ghafour, M. Y., Hama, H. K., Vo, B., & Khoshnevis, A. (2020). Multi-objective task and workflow scheduling approaches in cloud computing: a comprehensive review. Journal of Grid Computing, 18(3), 327-356.

[26] Xing, S., & Wang, Y. (2025). Proactive Data Placement in Heterogeneous Storage Systems via Predictive Multi-Objective Reinforcement Learning. IEEE Access.

[27] Lievens, F., Sackett, P. R., & De Corte, W. (2022). Weighting admission scores to balance predictiveness-diversity: the Pareto-optimization approach. Medical education, 56(2), 151-158.

[28] Cao, J., Zheng, W., Ge, Y., & Wang, J. (2025). DriftShield: Autonomous Fraud Detection via Actor-Critic Reinforcement Learning with Dynamic Feature Reweighting. IEEE Open Journal of the Computer Society.

[29] Wang, M., Zhang, X., Yang, Y., & Wang, J. (2025). Explainable Machine Learning in Risk Management: Balancing Accuracy and Interpretability. Journal of Financial Risk Management, 14(3), 185-198.