Q-Learning-Based Dynamic Spectrum Access in Cognitive Industrial Internet of Things

  title={Q-Learning-Based Dynamic Spectrum Access in Cognitive Industrial Internet of Things},
  author={Feng Li and Kwok-Yan Lam and Zhengguo Sheng and Xinggan Zhang and Kanglian Zhao and Li Wang},
  journal={Mobile Networks and Applications},
In recent years, Industrial Internet of Things (IIoT) has attracted growing attention from both academia and industry. Meanwhile, when traditional wireless sensor networks are applied to complex industrial field with high requirements for real time and robustness, how to design an efficient and practical cross-layer transmission mechanism needs to be fully investigated. In this paper, we propose a Q-learning-based dynamic spectrum access method for IIoT by introducing cognitive self-learning… 
Deep-Reinforcement-Learning-Based Spectrum Resource Management for Industrial Internet of Things
A new reward function is designed to drive the learning process, which takes into account the different communication requirements of various types of UEs, and the proposed algorithm can successfully achieve dynamic spectrum resource management in the IIoT network.
Dynamic Cooperative Spectrum Sensing Based on Deep Multi-User Reinforcement Learning
The deep multi-user reinforcement learning (DMRL) is proposed by introducing the cooperative strategy into dueling deep Q network (DDQN), which can achieve better performance on effectively enhancing spectrum utilization and reducing conflict rate in the dynamic cooperative spectrum sensing.
Spectrum access in cognitive IoT using reinforcement learning
A proposed proactive multiple channels spectrum access approach is introduced to enhance the spectrum access of CIoT through multiple available interfaces, whereinCIoT utilizes past channel states to predict the forthcoming spectrum availability.
Advances and Emerging Challenges in Cognitive Internet-of-Things
The structural frameworks and potential applications of cognitive IoT, including spectrum sensing, dynamic spectrum accessing, circumstantial perceiving, and self-learning, and the spectrum-based functionalities and heterogeneity for cognitive IoT are investigated.
A Q-Learning-Based Approach for Enhancing Energy Efficiency of Bluetooth Low Energy
A Q-learning-based scheduling algorithm is appropriately constructed to simultaneously provide a longer network lifetime and satisfy the QoS requirement and the numerical results show that the proposed Q- learning-based approach significantly increases the network lifetime compared to alternative methods while meeting QoS requirements.
Intelligent cognitive spectrum collaboration: Convergence of spectrum sensing, spectrum access, and coding technology
A joint optimization algorithm of dynamic spectrum access and coding is proposed and implemented using reinforcement learning, and the effectiveness of the algorithm is verified by simulations, thus providing a feasible research direction for the realization of cognitive spectrum collaboration.
Using Deep Q-Learning to Prolong the Lifetime of Correlated Internet of Things Devices
This paper proposes an updating mechanism leveraging Reinforcement Learning (RL) to take advantage of the exhibited correlation in the information collected, and implements the proposed updating mechanism employing deep Q-learning.
Reinforcement Learning-Based Control and Networking Co-Design for Industrial Internet of Things
This paper implements and integrates the reinforcement learning-based co-design approach on a realistic wireless cyber-physical simulator to conduct extensive experiments and demonstrates that the approach can effectively and quickly reconfigure the control and networking systems automatically in a dynamic industrial environment.
Energy-Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space
A deep reinforcement learning (DRL)-based scheduling mechanism capable of taking advantage of correlated information, which can significantly extend sensors’ lifetime and is compared to an idealized, all-knowing scheduler to demonstrate that its performance is near optimal.
Context Aware Data Perception in Cognitive Internet of Things - Cognitive Agent Approach
The experimental results show that the proposed method outperforms the cognitive agent approaches of data perception in terms of network lifetime, energy consumption, data perception accuracy, and throughput in the cognitive internet of things.


A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things
A new Q-learning-based transmission scheduling mechanism using deep learning for the CIoT is proposed to solve the problem of how to achieve the appropriate strategy to transmit packets of different buffers through multiple channels to maximize the system throughput.
Robust QoS-Aware Cross-layer Design of Adaptive Modulation Transmission on OFDM Systems in High-Speed Railway
An orthogonal frequency division multiplexing communication system that adopts frame-by-frame transmission in high-speed railway (HSR) scenario and a robust cross-layer transmission strategy that combines adaptive modulation (AM) scheme with truncated automatic repeat request protocol is proposed.
Interference Alignment Based on Antenna Selection With Imperfect Channel State Information in Cognitive Radio Networks
A novel IA scheme based on antenna selection (AS) to improve the received SINR of each user in IA-based CR networks and an efficient IA-AS algorithm based on discrete stochastic optimization (DSO) is proposed, which can converge quickly to the optimum with low computational complexity.
Adaptive Power Allocation Schemes for Spectrum Sharing in Interference-Alignment-Based Cognitive Radio Networks
Three PA algorithms to maximize the throughput of secondary users (SUs), the energy efficiency of the network, and the requirements of SUs, respectively, while guaranteeing the quality of service (QoS) of the primary user (PU) are proposed.
Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense Network
  • Min Chen, Yixue Hao
  • Computer Science
    IEEE Journal on Selected Areas in Communications
  • 2018
This paper investigates the task offloading problem in ultra-dense network aiming to minimize the delay while saving the battery life of user’s equipment and proposes an efficient offloading scheme which can reduce 20% of the task duration with 30% energy saving.
Superframe Planning and Access Latency of Slotted MAC for Industrial WSN in IoT Environment
IWSN designers can minimize the MAC access latency while satisfying the requirements at different generating rates of packet, number of nodes in the network, and packet buffer length of each node.
Data-Driven Computing and Caching in 5G Networks: Architecture and Delay Analysis
A novel network architecture using a resource cognitive engine and data engine, aimed at a global view of computing, caching, and communication resources in the network, and an optimal caching strategy for the small- cell cloud and the macro-cell cloud is proposed.
A review of industrial wireless networks in the context of Industry 4.0
This paper presents an overview of industrial WNs (IWNs), discusses IWN features and related techniques, and provides a new architecture based on quality of service and quality of data for IWNs.
Compressive Traffic Monitoring in Hybrid SDN
This work proposes a novel compressive traffic monitoring method for collecting real-time load information of all links that has better adaptability to the dynamic traffic changes and can reduce the maximal link usage by 39%.
A Cooperative Wireless Sensor Network for Indoor Industrial Monitoring
A cooperative WSN scheme is proposed by introducing a novel cooperation mechanism and a medium access control protocol that effectively increases the probability of correct decision about the state of the machine, reduces the probabilities of false alarms at a given signal level, and reduces the overall energy consumption as compared to noncooperative schemes.