Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach

@article{Shao2022LearningTC,
  title={Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach},
  author={Jiawei Shao and Yuyi Mao and Jun Zhang},
  journal={IEEE Journal on Selected Areas in Communications},
  year={2022},
  volume={40},
  pages={197-211}
}
This paper investigates task-oriented communication for edge inference, where a low-end edge device transmits the extracted feature vector of a local data sample to a powerful edge server for processing. It is critical to encode the data into an informative and compact representation for low-latency inference given the limited bandwidth. We propose a learning-based communication scheme that jointly optimizes feature extraction, source coding, and channel coding in a task-oriented manner, i.e… 
Task-Oriented Communication for Multi-Device Cooperative Edge Inference
TLDR
This paper proposes a learning-based communication scheme that optimizes local feature extraction and distributed feature encoding in a task-oriented manner and leverages an information bottleneck principle to extract the task-relevant feature at each edge device and adopt a distributed information bottleneck framework.
Communication-Computation Efficient Device-Edge Co-Inference via AutoML
TLDR
By selecting a suitable model split point and a pair of encoder/decoder for the intermediate feature vector, this problem is casted as a sequential decision problem, for which, a novel automated machine learning (AutoML) framework is proposed based on deep reinforcement learning (DRL).
Adaptable Semantic Compression and Resource Allocation for Task-Oriented Communications
TLDR
A deep learning-based task-oriented communication architecture is proposed where the user extracts, compresses and transmits semantics in an end-to-end (E2E) manner and an approach is proposed to compress the semantics according to their importance relevant to the task, namely, adaptable semantic compression (ASC).
Graph Neural Networks for Wireless Communications: From Theory to Practice
TLDR
For design guidelines, this paper proposes a unified framework that is applicable to general design problems in wireless networks, which includes graph modeling, neural architecture design, and theory-guided performance enhancement, and extensive simulations verify the theory and effectiveness of the proposed design framework.
How Neural Architectures Affect Deep Learning for Communication Networks?
TLDR
It is proved that by exploiting permutation invariance, a common property in communication networks, graph neural networks (GNNs) converge faster and generalize better than fully connected multi-layer perceptrons (MLPs), especially when the number of nodes is large.
Learn to Communicate with Neural Calibration: Scalability and Generalization
TLDR
The backbone of a traditional time-efficient algorithm is integrated with deep neural networks to achieve a high computational efficiency, while enjoying enhanced performance and the proposed neural calibration framework is applied to solve challenging resource management problems in massive multiple-input multiple-output (MIMO) systems.
Over-the-Air Ensemble Inference with Model Privacy
TLDR
Different over-the-air ensemble methods are introduced and it is shown that these schemes perform significantly better than their orthogonal counterparts, while using less resources and providing privacy guarantees.
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications
TLDR
The vision for scalable and trustworthy edge AI systems with integrated design of wireless communication strategies and decentralized machine learning models is provided and new design principles of wireless networks, service-driven resource allocation optimization methods, as well as a holistic end-to-end system architecture to support edge AI will be described.
Low-Complexity Recruitment for Collaborative Mobile Crowdsourcing Using Graph Neural Networks
TLDR
A novel low-complexity CMCS recruitment approach relying on graph neural networks (GNNs), specifically graph embedding and clustering techniques, to shrink the workers’ search space and afterwards, exploiting a metaheuristic genetic algorithm to select appropriate workers.
Deep Joint Source-Channel Coding for CSI Feedback: An End-to-End Approach
TLDR
A deep joint source- channel coding (DJSCC) based framework for the CSI feedback task that can simultaneously learn from the CSI source and the wireless channel and applies non-linear transform networks to compress the CSI.
...
1
2
...

References

SHOWING 1-10 OF 57 REFERENCES
On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views
This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and…
Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning
TLDR
This paper proposes an efficient and flexible 2-step pruning framework for DNN partition between mobile devices and edge servers that can greatly reduce either the wireless transmission workload of the device or the total computation workload.
Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
TLDR
Eggent is a collaborative and on-demand DNN co-inference framework with device-edge synergy that pursues two design knobs: DNN partitioning that adaptively partitions DNN computation between device and edge, in order to leverage hybrid computation resources in proximity for real-time DNN inference.
Neural Joint Source-Channel Coding
TLDR
This work proposes to jointly learn the encoding and decoding processes using a new discrete variational autoencoder model and obtains codes that are not only competitive against several separation schemes, but also learn useful robust representations of the data for downstream tasks such as classification.
Information Dropout: Learning Optimal Representations Through Noisy Computation
TLDR
It is proved that Information Dropout achieves a comparable or better generalization performance than binary dropout, especially on smaller models, since it can automatically adapt the noise to the structure of the network, as well as to the test sample.
Communication-Efficient Edge AI: Algorithms and Systems
TLDR
A comprehensive survey of the recent developments in various techniques for overcoming key communication challenges in edge AI systems is presented, and communication-efficient techniques are introduced from both algorithmic and system perspectives for training and inference tasks at the network edge.
BlockDrop: Dynamic Inference Paths in Residual Networks
TLDR
BlockDrop, an approach that learns to dynamically choose which layers of a deep network to execute during inference so as to best reduce total computation without degrading prediction accuracy, is introduced.
Branchy-GNN: A Device-Edge Co-Inference Framework for Efficient Point Cloud Processing
TLDR
This work proposes Branchy-GNN for efficient graph neural network (GNN) based point cloud processing by leveraging edge computing platforms and employs learning-based joint source-channel coding (JSCC) for the intermediate feature compression to reduce the communication overhead.
Once for All: Train One Network and Specialize it for Efficient Deployment
TLDR
This work proposes to train a once-for-all (OFA) network that supports diverse architectural settings by decoupling training and search, to reduce the cost and propose a novel progressive shrinking algorithm, a generalized pruning method that reduces the model size across many more dimensions than pruning.
Learning Structured Sparsity in Deep Neural Networks
TLDR
The results show that for CIFAR-10, regularization on layer depth can reduce 20 layers of a Deep Residual Network to 18 layers while improve the accuracy from 91.25% to 92.60%, which is still slightly higher than that of original ResNet with 32 layers.
...
1
2
3
4
5
...