Learning Automata Based Q-Learning for Content Placement in Cooperative Caching

@article{Yang2019LearningAB,
  title={Learning Automata Based Q-Learning for Content Placement in Cooperative Caching},
  author={Zhong Yang and Yuanwei Liu and Yue Chen and Lei Jiao},
  journal={IEEE Transactions on Communications},
  year={2019},
  volume={68},
  pages={3667-3680}
}
An optimization problem of content placement in cooperative caching is formulated, with the aim of maximizing the sum mean opinion score (MOS) of mobile users. Firstly, as user mobility and content popularity have significant impacts on the user experience, a recurrent neural network (RNN) is invoked for user mobility prediction and content popularity prediction. More particularly, practical data collected from GPS-tracker app on smartphones is tackled to test the accuracy of user mobility… 
View on IEEE
arxiv.org
20 Citations
Background Citations
13
Methods Citations
10

20 Citations

Cache-Aided NOMA Mobile Edge Computing: A Reinforcement Learning Approach

A novel non-orthogonal multiple access (NOMA) based cache-aided mobile edge computing (MEC) framework is proposed and it is proved that the BLA based action selection scheme is instantaneously self-correcting and the selected action is an optimal solution for each state.

Distributed Reinforcement Learning for NOMA-Enabled Mobile Edge Computing

In this article, the considered resource allocation problem is formulated as a long-term reward maximization problem that involves a joint optimization of task offloading decision, computation resource allocation, and caching decision.

Cooperative Edge Caching via Multi Agent Reinforcement Learning in Fog Radio Access Networks

Simulation results show that the proposed MARL-based cooperative caching scheme has remarkable performance compared with the benchmark schemes in minimizing the content transmission delay.

Towards Cost Minimization for Wireless Caching Networks With Recommendation and Uncharted Users’ Feature Information

A versatile long short term memory (LSTM) network assisted prediction paradigm is proposed to attain the preference schema of users with the assistance of their historical behavior data and it is rigorously proved the NP-hardness of obtaining the optimal recommendation and caching policies that jointly minimize the system cost.

Cooperative Edge Caching via Federated Deep Reinforcement Learning in Fog-RANs

A dueling deep Q-network based cooperative edge caching method to find the optimal caching policy in a distributed manner and horizontal federated learning is applied to address issues of over-consumption of resources during distributed training and data transmission process.

Effective Cache-Enabled Wireless Networks: An Artificial Intelligence- and Recommendation-Oriented Framework

This article presents the hierarchical, cache-enabled wireless network architecture, in which AI techniques and recommendation are utilized, respectively, to estimate users’ content requests in real time using historical data and to reshape users' content preference.

Deep Learning for Latent Events Forecasting in Content Caching Networks

Three machine learning models are proposed to predict latent events and events popularity, utilizing collected Twitter data with geo-tags and geographic information of the adjacent base stations (BSs) to enhance the caching efficiency.

Deep Learning for Latent Events Forecasting in Twitter Aided Caching Networks

A novel Twitter context aided content caching (TAC) framework is proposed for enhancing the caching efficiency by taking advantage of the legibility and massive volume of Twitter data and conserves superiority for natural language processing (NLP) in Twitter data.

Privacy-Preserving Federated Deep Learning for Cooperative Hierarchical Caching in Fog Computing

Experimental results demonstrate that FLCH outperforms five important baseline schemes in terms of the cache hit ratio, while preserving data privacy, and the results show the effectiveness of the proposed cooperative hierarchical caching mechanism for FLCH.

Revenue Maximization for Content-Oriented Wireless Caching Networks (CWCNs) With Repair and Recommendation Considerations

Numerical results confirm the convergence performance of the iterative algorithms and illustrate their revenue improvements compared to various baseline schemes.

References

SHOWING 1-10 OF 54 REFERENCES

Q-Learning for Content Placement in Wireless Cooperative Caching

An optimization problem of cooperative content caching is formulated, with the aim of maximizing the sum mean opinion score (MOS) of all users in the network, and a low complexity Q-learning based content caching algorithm is proposed which obtains a near-optimal solution.

Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities

In this paper, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown, providing a simple, yet practical asynchronous caching approach.

A Reinforcement-Learning Approach to Proactive Caching in Wireless Networks

This work proves the optimality of a threshold-based proactive caching scheme, which dynamically caches or removes appropriate contents from the memory, prior to being requested by the user, depending on the channel state, and proposes parametric representations for the threshold values and uses reinforcement-learning algorithms to find near-optimal parameterizations.

Context-Aware Proactive Content Caching With Service Differentiation in Wireless Networks

A novel algorithm for context-aware proactive caching that outperforms state-of-the-art algorithms in a real world data set, with an increase in the number of cache hits of at least 14%.

Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks

To model the two-way interactive influence between caching decisions at the parent and leaf nodes, a reinforcement learning (RL) framework is put forth and a scalable deep RL approach is pursued, which relies on a hyper-deep Q-network to learn the Q-function, and thus the optimal caching policy, in an online fashion.

Adaptive Caching via Deep Reinforcement Learning

To model the two-way interactive influence between caching decisions at the parent and leaf nodes, a reinforcement learning framework is put forth and the novel approach relies on a deep Q-network to learn the Q-function, and thus the optimal caching policy, in an online fashion.

Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks With Mobile Users

An algorithm that combines the machine learning framework of echo state networks (ESNs) with sublinear algorithms is proposed to solve the problem of proactive caching for cloud radio access networks (CRANs).

Cooperative Edge Caching in User-Centric Clustered Mobile Networks

Numerical results demonstrate that the proposed greedy content placement algorithm can reduce the average file transmission delay up to 45 percent compared with the non-cooperative and hit-ratio-maximal schemes.

Optimal Cooperative Content Caching and Delivery Policy for Heterogeneous Cellular Networks

Numerical results have shown that the proposed cooperative content caching and delivery policy can significantly improve content delivery performance in comparison with existing caching strategies.

Coding, Multicast, and Cooperation for Cache- Enabled Heterogeneous Small Cell Networks

In this paper, a typical cache-enabled small cell network under heterogeneous file and network settings is considered using maximum distance separable (MDS) codes for content restructuring, and a compound caching technique, which is referred to as multicast-aware cooperative caching, is developed.
...