Optimal Status Update for Caching Enabled IoT Networks: A Dueling Deep R-Network Approach

  title={Optimal Status Update for Caching Enabled IoT Networks: A Dueling Deep R-Network Approach},
  author={Chao Xu and Yiping Xie and Xijun Wang and H. Yang and D. Niyato and Tony Q. S. Quek},
In the Internet of Things (IoT) networks, caching is a promising technique to alleviate energy consumption of sensors by responding to users’ data requests with the data packets cached in the edge caching node (ECN). However, without an efficient status update strategy, the information obtained by users may be stale, which in return would inevitably deteriorate the accuracy and reliability of derived decisions for real-time applications. In this paper, we focus on striking the balance between… Expand
Optimizing the Long-Term Average Reward for Continuing MDPs: A Technical Report
This work has established a methodology for designing deep reinforcement learning (DRL) algorithms to maximize (resp. minimize) the average reward, by integrating R-learning, a tabular reinforcement learning algorithm tailored for maximizing the long-term average Reward, and traditional DRL algorithms, initially developed to optimize the discounted long- term cumulative reward rather the average one. Expand


AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks
A dynamic status update optimization problem to minimize the expectation of a long-term accumulative cost is formulated, which jointly considers the users' AoI and sensor's energy consumption, and a model-free reinforcement learning algorithm is proposed. Expand
Optimizing Information Freshness in Computing-Enabled IoT Networks
An analytical framework is developed to investigate the information freshness, in terms of peak age of information (PAoI), of a computing-enabled IoT system with multiple sensors, and a derivative-free algorithm is designed to find the optimal updating frequency. Expand
Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic
This paper model the cache update problem as a cooperative multi-agent Markov decision process with the goal of minimizing the long-term average weighted cost and devise a novel reinforcement learning approach, which is a discrete multi- agent variant of soft actor-critic (SAC). Expand
A novel caching mechanism for Internet of Things (IoT) sensing service with energy harvesting
This paper introduces a caching mechanism for the energy harvesting based Internet of Things (IoT) sensing service, and introduces the threshold adaptation algorithm that allows the sensing cache dynamically to adjust the parameter of caching to maximize the combined hit rate of the sensing service from multiple sensors. Expand
Cache in the air: exploiting content caching and delivery techniques for 5G systems
A novel edge caching scheme based on the concept of content-centric networking or information-centric networks is proposed and evaluated, using trace-driven simulations to evaluate the performance of the proposed scheme and validate the various advantages of the utilization of caching content in 5G mobile networks. Expand
Average AoI of Cached Status Updates for a Process Monitored by an Energy Harvesting Sensor
The average Age of Information (AoI) of the source seen at the aggregator is characterized as a function of the external request probability, the battery charging probability, and the probability that a fresh update will be generated by the EH sensor. Expand
A Reinforcement-Learning Approach to Proactive Caching in Wireless Networks
This work proves the optimality of a threshold-based proactive caching scheme, which dynamically caches or removes appropriate contents from the memory, prior to being requested by the user, depending on the channel state, and proposes parametric representations for the threshold values and uses reinforcement-learning algorithms to find near-optimal parameterizations. Expand
A Deep Reinforcement Learning Approach for Dynamic Contents Caching in HetNets
  • Manyou Ma, V. Wong
  • Computer Science, Engineering
  • ICC 2020 - 2020 IEEE International Conference on Communications (ICC)
  • 2020
A queue-aware cache content update scheduling algorithm based on Markov decision process (MDP) is developed to minimize the average AoI of the NNs delivered to the users plus the cost related to content updating. Expand
Minimum Age of Information in the Internet of Things With Non-Uniform Status Packet Sizes
  • Bo Zhou, W. Saad
  • Computer Science
  • IEEE Transactions on Wireless Communications
  • 2020
A low-complexity suboptimal policy is proposed through a semi-randomized base policy and linear approximated value functions and shown to exhibit a similar structure to the optimal policy, which provides a structural base for its effective performance. Expand
AoI-Delay Tradeoff in Mobile Edge Caching With Freshness-Aware Content Refreshing
The results indicate that the proposed scheme can restrain frequent refreshing as the request arrival rate increases, whereby the average delay can be reduced by around 80% while maintaining the AoI below one second in heavily-loaded scenarios. Expand