Learning for Video Compression
@article{Chen2020LearningFV, title={Learning for Video Compression}, author={Zhibo Chen and Tianyu He and Xin Jin and Feng Wu}, journal={IEEE Transactions on Circuits and Systems for Video Technology}, year={2020}, volume={30}, pages={566-576} }
One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video…
Figures and Tables from this paper
71 Citations
DVC: An End-To-End Deep Video Compression Framework
- Computer Science2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019
This paper proposes the first end-to-end video compression deep model that jointly optimizes all the components for video compression, and shows that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard MS-SSIM.
Variable Rate Video Compression using a Hybrid Recurrent Convolutional Learning Framework
- Computer Science2020 International Conference on Computer Communication and Informatics (ICCCI)
- 2020
PredEncoder is presented, a hybrid video compression framework based on the concept of predictive auto-encoding that models the temporal correlations between consecutive video frames using a prediction network which is then combined with a progressive encoder network to exploit the spatial redundancies.
Comprehensive Analysis of Flow Incorporated Neural Network based Lightweight Video Compression Architecture
- Computer Science
- 2021
A deep learning based video compression architecture has been proposed comprises of frame autoencoder, flow autoen coder and motion extension network for the reconstruction of predicted frames and results in significant improvement in visual perception quality measured in SSIM and PSNR when compared to some state-of-art techniques.
Deep Learning Approach to Video Compression
- Computer Science2019 IEEE Bombay Section Signature Conference (IBSSC)
- 2019
A method for video compression using neural networks that outperforms the H.264/AVC video coding standard as measured using Multi-Scale - Structural Similarity Index (MS-SSIM).
Motion-aware deep video coding network
- Computer ScienceDefense + Commercial Sensing
- 2020
This work proposes a novel motion-aware deep video coding network, in which inter-frame correlations are effectively explored via a block-level motion compensation network, which significantly improves the decoding quality under the same compression ratio.
End-to-End Learning of Video Compression using Spatio-Temporal Autoencoders
- Computer Science2020 IEEE Workshop on Signal Processing Systems (SiPS)
- 2020
This work formalizes it as the rate-distortion optimization of a single spatio-temporal autoencoder, by jointly learning a latent-space projection transform, and a synthesis transform for low-bitrate video compression, and shows competitive performance against standard codecs.
Learning to Compress Videos without Computing Motion
- Computer ScienceSignal Process. Image Commun.
- 2022
Learned Video Compression with Feature-level Residuals
- Computer Science2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- 2020
This paper presents an end-to-end video compression network for P-frame challenge on CLIC, and combines the advantages of both pixel-level and feature-level residual compression methods by model ensembling.
VIDEO COMPRESSION USING (END-TO-END) DEEP LEARNING
- Computer Science
- 2018
This dissertation proposes what it is believed to be the first the first approach to end-to-end learning of a single network for video compression, and shows better performance than the baseline and the MPEG-4 Part 2 codec, being competitive with H.264/AVC and H.265/HEVC for low bitrates.
Learning Cross-Scale Prediction for Efficient Neural Video Compression
- Computer ScienceArXiv
- 2021
This paper presents the first neural video codec that can compete with the latest coding standard H.266/VVC in terms of sRGB PSNR on UVG dataset for the low-latency mode and proposes a novel cross-scale prediction module that achieves more effective motion compensation.
References
SHOWING 1-10 OF 61 REFERENCES
Video Compression - From Concepts to the H.264/AVC Standard
- Computer ScienceProceedings of the IEEE
- 2005
This paper starts with an explanation of the basic concepts of video codec design and then explains how various features have been integrated into international standards, up to and including the most recent such standard, known as H.264/AVC.
A Novel Deep Learning-Based Method of Improving Coding Efficiency from the Decoder-End for HEVC
- Computer Science2017 Data Compression Conference (DCC)
- 2017
A very deep convolutional neural network is proposed to automatically remove the artifacts and enhance the details of HEVC-compressed videos, by utilizing that underused information left in the bit-streams and external images.
A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
- Computer ScienceMMM
- 2017
A CNN-based post-processing algorithm for High Efficiency Video Coding (HEVC), the state-of-the-art video coding standard, that outperforms previously studied networks in achieving higher bit-rate reduction, lower memory cost, and multiplied computational speedup.
A convolutional neural network approach for half-pel interpolation in video coding
- Computer Science2017 IEEE International Symposium on Circuits and Systems (ISCAS)
- 2017
This work proposes to design a CNN-based interpolation filter (CNNIF) for video coding, Inspired by the great success of convolutional neural network (CNN) in computer vision, which achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration.
A deep convolutional neural network approach for complexity reduction on intra-mode HEVC
- Computer Science2017 IEEE International Conference on Multimedia and Expo (ICME)
- 2017
A complexity reduction approach for intra-mode HEVC, which learns a deep convolutional neural network model to predict CTU partition instead of RDO, which reduces intramode encoding time by 62.25% and 69.06% with negligible Bj⊘ntegaard delta bit-rate.
Generative Compression
- Computer Science2018 Picture Coding Symposium (PCS)
- 2018
The concept of generative compression, the compression of data using generative models, is described and it is suggested that it is a direction worth pursuing to produce more accurate and visually pleasing reconstructions at deeper compression levels for both image and video data.
Learning to Inpaint for Image Compression
- Computer ScienceNIPS
- 2017
Predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and learning to inpaint before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation.
Neural network-based arithmetic coding of intra prediction modes in HEVC
- Computer Science2017 IEEE Visual Communications and Image Processing (VCIP)
- 2017
This paper proposes an arithmetic coding strategy by training neural networks, and makes preliminary studies on coding of the intra prediction modes in HEVC.
An End-to-End Compression Framework Based on Convolutional Neural Networks
- Computer Science2017 Data Compression Conference (DCC)
- 2017
A unified end-to-end learning framework is developed to simultaneously learn CrCNN and ReCNN such that the compact representation obtained by CrCNN preserves the structural information of the image, which facilitates to accurately reconstruct the decoded image using ReCNN and also makes the proposed compression framework compatible with existing image coding standards.
Lossy Image Compression with Compressive Autoencoders
- Computer ScienceICLR
- 2017
It is shown that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs, and furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images.