FVC: A New Framework towards Deep Video Compression in Feature Space

@article{Hu2021FVCAN,
  title={FVC: A New Framework towards Deep Video Compression in Feature Space},
  author={Zhihao Hu and Guo Lu and Dong Xu},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={1502-1511}
}
  • Zhihao Hu, Guo Lu, Dong Xu
  • Published 20 May 2021
  • Engineering, Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Learning based video compression attracts increasing attention in the past few years. The previous hybrid coding approaches rely on pixel space operations to reduce spatial and temporal redundancy, which may suffer from inaccurate motion estimation or less effective motion compensation. In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature… Expand

Figures and Tables from this paper

Deep Learning for Visual Data Compression
  • Guo Lu, Ren Yang, Shenlong Wang, Shan Liu, R. Timofte
  • Computer Science
  • ACM Multimedia
  • 2021
TLDR
The recent progress in deep learning based visual data compression, including image compression, video compression and point cloud compression is introduced. Expand
Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set
TLDR
A video compression algorithm based on instance-adaptive learning that enables a competitive performance even after reducing the network size by 72 % and reduces the capacity requirements on compression models is introduced. Expand
Self-Conditioned Probabilistic Learning of Video Rescaling
  • Yuan Tian, Guo Lu, +4 authors Zhiyong Gao
  • Computer Science
  • ArXiv
  • 2021
TLDR
A self-conditioned probabilistic framework for video rescaling to learn the paired downscaling and upscaling procedures simultaneously, and extends the framework to a lossy video compression system, in which a gradient estimator for non-differential industrial lossy codecs is proposed for the end-to-end training of the whole system. Expand
Deep Video Coding with Dual-Path Generative Adversarial Network
  • Tiesong Zhao, Weize Feng, Hongji Zeng, Yuzhen Niu, Jiaying Liu
  • Computer Science, Engineering
  • ArXiv
  • 2021
The deep-learning-based video coding has attracted substantial attention for its great potential to squeeze out the spatial-temporal redundancies of video sequences. This paper proposes an efficientExpand
DeepWiVe: Deep-Learning-Aided Wireless Video Transmission
We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals toExpand
Temporal Context Mining for Learned Video Compression
  • Xihua Sheng, Jiahao Li, Bin Li, Li Li, Dong Liu, Yan Lu
  • Computer Science, Engineering
  • 2021
We address end-to-end learned video compression with a special focus on better learning and utilizing temporal contexts. For temporal context mining, we propose to store not only the previouslyExpand

References

SHOWING 1-10 OF 43 REFERENCES
M-LVC: Multiple Frames Prediction for Learned Video Compression
TLDR
An end-to-end learned video compression scheme for low-latency scenarios that introduces the usage of the previous multiple frames as references and designs a MV refinement network and a residual refinement network, taking use of the multiple reference frames as well. Expand
Improving Deep Video Compression by Resolution-adaptive Flow Coding
TLDR
A new framework called Resolution-adaptive Flow Coding (RaFC) is proposed to effectively compress the flow maps globally and locally, in which multi-resolution representations are used for both the input flow maps and the output motion features of the MV encoder. Expand
An End-to-End Learning Framework for Video Compression
TLDR
This paper proposes the first end-to-end deep video compression framework that can outperform the widely used video coding standard H.264 and be even on par with the latest standard H265. Expand
Neural Inter-Frame Compression for Video Coding
TLDR
This work presents an inter-frame compression approach for neural video coding that can seamlessly build up on different existing neural image codecs and proposes to compute residuals directly in latent space instead of in pixel space to reuse the same image compression network for both key frames and intermediate frames. Expand
DVC: An End-To-End Deep Video Compression Framework
TLDR
This paper proposes the first end-to-end video compression deep model that jointly optimizes all the components for video compression, and shows that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard MS-SSIM. Expand
Learned Video Compression with Feature-level Residuals
TLDR
This paper presents an end-to-end video compression network for P-frame challenge on CLIC, and combines the advantages of both pixel-level and feature-level residual compression methods by model ensembling. Expand
Content Adaptive and Error Propagation Aware Deep Video Compression
TLDR
This work proposes a content adaptive and error propagation aware video compression system that outperforms the state-of-the-art learning based video codecs on benchmark datasets without increasing the model size or decreasing the decoding speed. Expand
Learning Image and Video Compression Through Spatial-Temporal Energy Compaction
TLDR
This paper presents an image compression architecture using a convolutional autoencoder, and then generalizes image compression to video compression, by adding an interpolation loop into both encoder and decoder sides, to achieve higher image compression performance. Expand
EDVR: Video Restoration With Enhanced Deformable Convolutional Networks
TLDR
This work proposes a novel Video Restoration framework with Enhanced Deformable convolutions, termed EDVR, and proposes a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. Expand
Scale-Space Flow for End-to-End Optimized Video Compression
TLDR
This paper proposes scale-space flow, an intuitive generalization of optical flow that adds a scale parameter to allow the network to better model uncertainty and outperform analogous state-of-the art learned video compression models while being trained using a much simpler procedure and without any pre-trained optical flow networks. Expand
...
1
2
3
4
5
...