Learning for Video Compression

@article{Chen2020LearningFV,
  title={Learning for Video Compression},
  author={Zhibo Chen and Tianyu He and X. Jin and Feng Wu},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2020},
  volume={30},
  pages={566-576}
}
  • Zhibo Chen, Tianyu He, +1 author Feng Wu
  • Published 2020
  • Computer Science, Engineering
  • IEEE Transactions on Circuits and Systems for Video Technology
One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video… Expand
View on IEEE
arxiv.org
55 Citations
Highly Influential Citations
1
Background Citations
24
Methods Citations
17
Results Citations
2

55 Citations

Citation Type

Citation Type

DVC: An End-To-End Deep Video Compression Framework
TLDR
This paper proposes the first end-to-end video compression deep model that jointly optimizes all the components for video compression, and shows that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard MS-SSIM. Expand
Comprehensive Analysis of Flow Incorporated Neural Network based Lightweight Video Compression Architecture
TLDR
A deep learning based video compression architecture has been proposed comprises of frame autoencoder, flow autoen coder and motion extension network for the reconstruction of predicted frames and results in significant improvement in visual perception quality measured in SSIM and PSNR when compared to some state-of-art techniques. Expand
Deep Learning Approach to Video Compression
TLDR
A method for video compression using neural networks that outperforms the H.264/AVC video coding standard as measured using Multi-Scale - Structural Similarity Index (MS-SSIM). Expand
Motion-aware deep video coding network
  • R. Khan, Y. Liu
  • Engineering, Computer Science
  • Defense + Commercial Sensing
  • 2020
TLDR
This work proposes a novel motion-aware deep video coding network, in which inter-frame correlations are effectively explored via a block-level motion compensation network, which significantly improves the decoding quality under the same compression ratio. Expand
End-to-End Learning of Video Compression using Spatio-Temporal Autoencoders
TLDR
This work formalizes it as the rate-distortion optimization of a single spatio-temporal autoencoder, by jointly learning a latent-space projection transform, and a synthesis transform for low-bitrate video compression, and shows competitive performance against standard codecs. Expand
Learned Video Compression with Feature-level Residuals
TLDR
This paper presents an end-to-end video compression network for P-frame challenge on CLIC, and combines the advantages of both pixel-level and feature-level residual compression methods by model ensembling. Expand
VIDEO COMPRESSION USING (END-TO-END) DEEP LEARNING
Deep learning (DL) is having a revolutionary impact in image processing, with DL-based approaches now holding the state of the art in many tasks, including image compression. However, video… Expand
Neural Generation of Blocks for Video Coding
TLDR
This paper proposes encoding content-specific learned parameters of a GNN within a video bitstream at specific times and using the GNN to generate content for specific ranges of blocks and frames, which maximizes the usefulness of the information contained in the learned parameters. Expand
Improving Deep Video Compression by Resolution-adaptive Flow Coding
TLDR
A new framework called Resolution-adaptive Flow Coding (RaFC) is proposed to effectively compress the flow maps globally and locally, in which multi-resolution representations are used for both the input flow maps and the output motion features of the MV encoder. Expand
Content Adaptive and Error Propagation Aware Deep Video Compression
TLDR
This work proposes a content adaptive and error propagation aware video compression system that outperforms the state-of-the-art learning based video codecs on benchmark datasets without increasing the model size or decreasing the decoding speed. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 61 REFERENCES
Video Compression - From Concepts to the H.264/AVC Standard
TLDR
This paper starts with an explanation of the basic concepts of video codec design and then explains how various features have been integrated into international standards, up to and including the most recent such standard, known as H.264/AVC. Expand
A Novel Deep Learning-Based Method of Improving Coding Efficiency from the Decoder-End for HEVC
TLDR
A very deep convolutional neural network is proposed to automatically remove the artifacts and enhance the details of HEVC-compressed videos, by utilizing that underused information left in the bit-streams and external images. Expand
A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
TLDR
A CNN-based post-processing algorithm for High Efficiency Video Coding (HEVC), the state-of-the-art video coding standard, that outperforms previously studied networks in achieving higher bit-rate reduction, lower memory cost, and multiplied computational speedup. Expand
A convolutional neural network approach for half-pel interpolation in video coding
TLDR
This work proposes to design a CNN-based interpolation filter (CNNIF) for video coding, Inspired by the great success of convolutional neural network (CNN) in computer vision, which achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration. Expand
A deep convolutional neural network approach for complexity reduction on intra-mode HEVC
TLDR
A complexity reduction approach for intra-mode HEVC, which learns a deep convolutional neural network model to predict CTU partition instead of RDO, which reduces intramode encoding time by 62.25% and 69.06% with negligible Bj⊘ntegaard delta bit-rate. Expand
Generative Compression
TLDR
The concept of generative compression, the compression of data using generative models, is described and it is suggested that it is a direction worth pursuing to produce more accurate and visually pleasing reconstructions at deeper compression levels for both image and video data. Expand
Learning to Inpaint for Image Compression
TLDR
Predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and learning to inpaint before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation. Expand
Neural network-based arithmetic coding of intra prediction modes in HEVC
TLDR
This paper proposes an arithmetic coding strategy by training neural networks, and makes preliminary studies on coding of the intra prediction modes in HEVC. Expand
An End-to-End Compression Framework Based on Convolutional Neural Networks
TLDR
A unified end-to-end learning framework is developed to simultaneously learn CrCNN and ReCNN such that the compact representation obtained by CrCNN preserves the structural information of the image, which facilitates to accurately reconstruct the decoded image using ReCNN and also makes the proposed compression framework compatible with existing image coding standards. Expand
Lossy Image Compression with Compressive Autoencoders
TLDR
It is shown that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs, and furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images. Expand
...
1
2
3
4
5
...