Autonomic decentralized elasticity based on a reinforcement learning controller for cloud applications

@article{Nouri2019AutonomicDE,
  title={Autonomic decentralized elasticity based on a reinforcement learning controller for cloud applications},
  author={Seyed Mohammad Reza Nouri and Han Li and Srikumar Venugopal and Wenxia Guo and MingYun He and Wenhong Tian},
  journal={Future Gener. Comput. Syst.},
  year={2019},
  volume={94},
  pages={765-780}
}
  • S. Nouri, Han Li, W. Tian
  • Published 1 May 2019
  • Computer Science
  • Future Gener. Comput. Syst.
Self-adaptive Threshold-based Policy for Microservices Elasticity
TLDR
This paper proposes MB Threshold and QL Threshold, two policies that employ respectively model-based and model-free reinforcement learning approaches to learn threshold update strategies, and shows the benefits and flexibility of the proposed solution.
A Deep Q-learning Scaling Policy for Elastic Application Deployment
TLDR
Deep Q-learning is proposed, a model-free RL solution that uses a deep neural network to approximate the system dynamics and shows the benefits and flexibility of Deep Q- learning with respect toQ-learning in scaling applications.
Reinforcement Learning-based Autoscaling of Workflows in the Cloud: A Survey
TLDR
This work surveys exhaustively those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies for autoscaling in Cloud, and discusses open problems and provides a prospect of future research in the area.
Efficient autonomic and elastic resource management techniques in cloud environment: taxonomy and analysis
TLDR
A comprehensive review along with a taxonomy of the most recent existing autonomic and elastic RM techniques in a cloud environment is presented to ensure the Quality-of-Service (QoS) of cloud-based applications, satisfy the cloud user requirements, and avoid a Service-Level Agreement (SLA) violations.
Web Auto Configuration for N-Tier in VM based Dynamic Environment by Reinforcement Learning Approach: A Study
TLDR
The studied methodology of machine learning is described, which will guide how optimize all the parameters with the best results in terms of web usability.
Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions
TLDR
This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of D RL in scheduling of Cloud computing.
ControCity: An Autonomous Approach for Controlling Elasticity Using Buffer Management in Cloud Computing Environment
TLDR
Controlling Elasticity (ControCity) framework for controlling resources elasticity through using “buffer management ” and “elasticity management” and experimental results indicate that the ControCity reduces the response time, and increases the resource utilization and elasticity by up to 8.4% and 5.4%, respectively, compared with the other approaches.
Hierarchical Scaling of Microservices in Kubernetes
TLDR
This paper presents me-kube (Multi-level Elastic Kubernetes), a Kubernetses extension that introduces a hierarchical architecture for controlling the elasticity of microservice-based applications, and proposes novel proactive and reactive hierarchical control policies, based on queuing theory.
...
...

References

SHOWING 1-10 OF 43 REFERENCES
Using reinforcement learning for controlling an elastic web application hosting platform
TLDR
Reinforcement learning is used to orchestrate control actions such as provisioning servers and application placement to meet performance requirements and minimize ongoing costs in a distributed, elastic hosting architecture.
Towards autonomic workload provisioning for enterprise Grids and clouds
TLDR
A decentralized, robust online clustering approach that addresses the distributed nature of these environments, and can be used to detect patterns and trends, and use this information to optimize provisioning of virtual (VM) resources.
Online Resource Allocation Using Decompositional Reinforcement Learning
TLDR
Early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory, and with better scalability than previous approaches.
Autonomous resource provisioning for multi-service web applications
TLDR
This work proposes to assign an SLA only to the front-end service, and demonstrates through extensive experiments that the system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization.
Appliance-Based Autonomic Provisioning Framework for Virtualized Outsourcing Data Center
TLDR
Experimental results based on a prototype system demonstrate that system-level performance has been greatly improved by taking advantage of fine-grained server consolidation and the efficiency of the framework through significantly reducing provisioning time.
A scalable application placement controller for enterprise data centers
TLDR
This paper proposes a new algorithm that can produce within 30seconds high-quality solutions for hard placement problems with thousands of machines and thousands of applications, and has been implemented and adopted in a leading commercial middleware product for managing the performance of Web applications.
A Cost-Aware Elasticity Provisioning System for the Cloud
In this paper we present Kingfisher, a {\em cost-aware} system that provides efficient support for elasticity in the cloud by (i) leveraging multiple mechanisms to reduce the time to transition to
Autonomic resource management in virtualized data centers using fuzzy logic-based approaches
TLDR
Experimental results obtained through a prototype implementation demonstrate that the proposed two-level resource management system can significantly reduce resource consumption while still achieving application performance targets.
Efficient Server Provisioning with Control for End-to-End Response Time Guarantee on Multitier Clusters
  • P. Lama, Xiaobo Zhou
  • Computer Science
    IEEE Transactions on Parallel and Distributed Systems
  • 2012
TLDR
An efficient server provisioning approach based on an end-to-end resource allocation optimization model integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90th-percentile end- to-end response time guarantees on multitier clusters.
Utility functions in autonomic systems
TLDR
A distributed architecture, implemented in a realistic prototype data center, that demonstrates how utility functions can enable a collection of autonomic elements to continually optimize the use of computational resources in a dynamic, heterogeneous environment is presented.
...
...