Learning for Video Compression

@article{Chen2020LearningFV,
  title={Learning for Video Compression},
  author={Zhibo Chen and Tianyu He and Xin Jin and Feng Wu},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2020},
  volume={30},
  pages={566-576}
}
  • Zhibo Chen, Tianyu He, Feng Wu
  • Published 26 April 2018
  • Computer Science
  • IEEE Transactions on Circuits and Systems for Video Technology
One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video… 
[PDF] Semantic Reader
71 Citations
Highly Influential Citations
1
Background Citations
33
Methods Citations
21
Results Citations
2

Figures and Tables from this paper

71 Citations

DVC: An End-To-End Deep Video Compression Framework
TLDR
This paper proposes the first end-to-end video compression deep model that jointly optimizes all the components for video compression, and shows that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard MS-SSIM.
Variable Rate Video Compression using a Hybrid Recurrent Convolutional Learning Framework
  • Aishwarya Jadhav
  • Computer Science
    2020 International Conference on Computer Communication and Informatics (ICCCI)
  • 2020
TLDR
PredEncoder is presented, a hybrid video compression framework based on the concept of predictive auto-encoding that models the temporal correlations between consecutive video frames using a prediction network which is then combined with a progressive encoder network to exploit the spatial redundancies.
Comprehensive Analysis of Flow Incorporated Neural Network based Lightweight Video Compression Architecture
TLDR
A deep learning based video compression architecture has been proposed comprises of frame autoencoder, flow autoen coder and motion extension network for the reconstruction of predicted frames and results in significant improvement in visual perception quality measured in SSIM and PSNR when compared to some state-of-art techniques.
Deep Learning Approach to Video Compression
TLDR
A method for video compression using neural networks that outperforms the H.264/AVC video coding standard as measured using Multi-Scale - Structural Similarity Index (MS-SSIM).
Motion-aware deep video coding network
TLDR
This work proposes a novel motion-aware deep video coding network, in which inter-frame correlations are effectively explored via a block-level motion compensation network, which significantly improves the decoding quality under the same compression ratio.
End-to-End Learning of Video Compression using Spatio-Temporal Autoencoders
TLDR
This work formalizes it as the rate-distortion optimization of a single spatio-temporal autoencoder, by jointly learning a latent-space projection transform, and a synthesis transform for low-bitrate video compression, and shows competitive performance against standard codecs.
Learning to Compress Videos without Computing Motion
Learned Video Compression with Feature-level Residuals
TLDR
This paper presents an end-to-end video compression network for P-frame challenge on CLIC, and combines the advantages of both pixel-level and feature-level residual compression methods by model ensembling.
VIDEO COMPRESSION USING (END-TO-END) DEEP LEARNING
TLDR
This dissertation proposes what it is believed to be the first the first approach to end-to-end learning of a single network for video compression, and shows better performance than the baseline and the MPEG-4 Part 2 codec, being competitive with H.264/AVC and H.265/HEVC for low bitrates.
Learning Cross-Scale Prediction for Efficient Neural Video Compression
TLDR
This paper presents the first neural video codec that can compete with the latest coding standard H.266/VVC in terms of sRGB PSNR on UVG dataset for the low-latency mode and proposes a novel cross-scale prediction module that achieves more effective motion compensation.
...
...

References

SHOWING 1-10 OF 61 REFERENCES
Video Compression - From Concepts to the H.264/AVC Standard
TLDR
This paper starts with an explanation of the basic concepts of video codec design and then explains how various features have been integrated into international standards, up to and including the most recent such standard, known as H.264/AVC.
A Novel Deep Learning-Based Method of Improving Coding Efficiency from the Decoder-End for HEVC
TLDR
A very deep convolutional neural network is proposed to automatically remove the artifacts and enhance the details of HEVC-compressed videos, by utilizing that underused information left in the bit-streams and external images.
A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
TLDR
A CNN-based post-processing algorithm for High Efficiency Video Coding (HEVC), the state-of-the-art video coding standard, that outperforms previously studied networks in achieving higher bit-rate reduction, lower memory cost, and multiplied computational speedup.
A convolutional neural network approach for half-pel interpolation in video coding
TLDR
This work proposes to design a CNN-based interpolation filter (CNNIF) for video coding, Inspired by the great success of convolutional neural network (CNN) in computer vision, which achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration.
A deep convolutional neural network approach for complexity reduction on intra-mode HEVC
TLDR
A complexity reduction approach for intra-mode HEVC, which learns a deep convolutional neural network model to predict CTU partition instead of RDO, which reduces intramode encoding time by 62.25% and 69.06% with negligible Bj⊘ntegaard delta bit-rate.
Generative Compression
TLDR
The concept of generative compression, the compression of data using generative models, is described and it is suggested that it is a direction worth pursuing to produce more accurate and visually pleasing reconstructions at deeper compression levels for both image and video data.
Learning to Inpaint for Image Compression
TLDR
Predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and learning to inpaint before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation.
Neural network-based arithmetic coding of intra prediction modes in HEVC
TLDR
This paper proposes an arithmetic coding strategy by training neural networks, and makes preliminary studies on coding of the intra prediction modes in HEVC.
An End-to-End Compression Framework Based on Convolutional Neural Networks
TLDR
A unified end-to-end learning framework is developed to simultaneously learn CrCNN and ReCNN such that the compact representation obtained by CrCNN preserves the structural information of the image, which facilitates to accurately reconstruct the decoded image using ReCNN and also makes the proposed compression framework compatible with existing image coding standards.
Lossy Image Compression with Compressive Autoencoders
TLDR
It is shown that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs, and furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images.
...
...