# Autonomic decentralized elasticity based on a reinforcement learning controller for cloud applications

@article{Nouri2019AutonomicDE,
title={Autonomic decentralized elasticity based on a reinforcement learning controller for cloud applications},
author={Seyed Mohammad Reza Nouri and Han Li and Srikumar Venugopal and Wenxia Guo and MingYun He and Wenhong Tian},
journal={Future Gener. Comput. Syst.},
year={2019},
volume={94},
pages={765-780}
}
• S. Nouri, Han Li
• Published 1 May 2019
• Computer Science
• Future Gener. Comput. Syst.
Self-adaptive Threshold-based Policy for Microservices Elasticity
• Computer Science
2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)
• 2020
This paper proposes MB Threshold and QL Threshold, two policies that employ respectively model-based and model-free reinforcement learning approaches to learn threshold update strategies, and shows the benefits and flexibility of the proposed solution.
A Deep Q-learning Scaling Policy for Elastic Application Deployment
Deep Q-learning is proposed, a model-free RL solution that uses a deep neural network to approximate the system dynamics and shows the benefits and flexibility of Deep Q- learning with respect toQ-learning in scaling applications.
Reinforcement Learning-based Autoscaling of Workflows in the Cloud: A Survey
• Computer Science
ArXiv
• 2020
This work surveys exhaustively those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies for autoscaling in Cloud, and discusses open problems and provides a prospect of future research in the area.
Efficient autonomic and elastic resource management techniques in cloud environment: taxonomy and analysis
• Computer Science
Wirel. Networks
• 2021
A comprehensive review along with a taxonomy of the most recent existing autonomic and elastic RM techniques in a cloud environment is presented to ensure the Quality-of-Service (QoS) of cloud-based applications, satisfy the cloud user requirements, and avoid a Service-Level Agreement (SLA) violations.
Web Auto Configuration for N-Tier in VM based Dynamic Environment by Reinforcement Learning Approach: A Study
• Computer Science
Computer Science &amp; Engineering: An International Journal
• 2022
The studied methodology of machine learning is described, which will guide how optimize all the parameters with the best results in terms of web usability.
Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions
• Computer Science
ArXiv
• 2021
This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of D RL in scheduling of Cloud computing.
ControCity: An Autonomous Approach for Controlling Elasticity Using Buffer Management in Cloud Computing Environment
• Computer Science
IEEE Access
• 2019
Controlling Elasticity (ControCity) framework for controlling resources elasticity through using “buffer management ” and “elasticity management” and experimental results indicate that the ControCity reduces the response time, and increases the resource utilization and elasticity by up to 8.4% and 5.4%, respectively, compared with the other approaches.
Hierarchical Scaling of Microservices in Kubernetes
• Computer Science
2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)
• 2020
This paper presents me-kube (Multi-level Elastic Kubernetes), a Kubernetses extension that introduces a hierarchical architecture for controlling the elasticity of microservice-based applications, and proposes novel proactive and reactive hierarchical control policies, based on queuing theory.

## References

SHOWING 1-10 OF 43 REFERENCES
Using reinforcement learning for controlling an elastic web application hosting platform
• Computer Science
ICAC '11
• 2011
Reinforcement learning is used to orchestrate control actions such as provisioning servers and application placement to meet performance requirements and minimize ongoing costs in a distributed, elastic hosting architecture.
Towards autonomic workload provisioning for enterprise Grids and clouds
• Computer Science
2009 10th IEEE/ACM International Conference on Grid Computing
• 2009
A decentralized, robust online clustering approach that addresses the distributed nature of these environments, and can be used to detect patterns and trends, and use this information to optimize provisioning of virtual (VM) resources.
Online Resource Allocation Using Decompositional Reinforcement Learning
Early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory, and with better scalability than previous approaches.
Autonomous resource provisioning for multi-service web applications
• Computer Science
WWW '10
• 2010
This work proposes to assign an SLA only to the front-end service, and demonstrates through extensive experiments that the system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization.
Appliance-Based Autonomic Provisioning Framework for Virtualized Outsourcing Data Center
• Computer Science
Fourth International Conference on Autonomic Computing (ICAC'07)
• 2007
Experimental results based on a prototype system demonstrate that system-level performance has been greatly improved by taking advantage of fine-grained server consolidation and the efficiency of the framework through significantly reducing provisioning time.
A scalable application placement controller for enterprise data centers
• Computer Science
WWW '07
• 2007
This paper proposes a new algorithm that can produce within 30seconds high-quality solutions for hard placement problems with thousands of machines and thousands of applications, and has been implemented and adopted in a leading commercial middleware product for managing the performance of Web applications.
A Cost-Aware Elasticity Provisioning System for the Cloud
• Computer Science
2011 31st International Conference on Distributed Computing Systems
• 2011
In this paper we present Kingfisher, a {\em cost-aware} system that provides efficient support for elasticity in the cloud by (i) leveraging multiple mechanisms to reduce the time to transition to
Autonomic resource management in virtualized data centers using fuzzy logic-based approaches
• Computer Science
Cluster Computing
• 2008
Experimental results obtained through a prototype implementation demonstrate that the proposed two-level resource management system can significantly reduce resource consumption while still achieving application performance targets.
Efficient Server Provisioning with Control for End-to-End Response Time Guarantee on Multitier Clusters
• Computer Science
IEEE Transactions on Parallel and Distributed Systems
• 2012
An efficient server provisioning approach based on an end-to-end resource allocation optimization model integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90th-percentile end- to-end response time guarantees on multitier clusters.
Utility functions in autonomic systems
• Computer Science
International Conference on Autonomic Computing, 2004. Proceedings.
• 2004
A distributed architecture, implemented in a realistic prototype data center, that demonstrates how utility functions can enable a collection of autonomic elements to continually optimize the use of computational resources in a dynamic, heterogeneous environment is presented.