Video Compression through Image Interpolation

@inproceedings{Wu2018VideoCT,
  title={Video Compression through Image Interpolation},
  author={Chao-Yuan Wu and Nayan Singhal and Philipp Kr{\"a}henb{\"u}hl},
  booktitle={ECCV},
  year={2018}
}
An ever increasing amount of our digital communication, media consumption, and content creation revolves around videos. We share, watch, and archive many aspects of our lives through them, all of which are powered by strong video compression. Traditional video compression is laboriously hand designed and hand optimized. This paper presents an alternative in an end-to-end deep learning codec. Our codec builds on one simple idea: Video compression is repeated image interpolation. It thus benefits… 

Learning Image and Video Compression Through Spatial-Temporal Energy Compaction

TLDR
This paper presents an image compression architecture using a convolutional autoencoder, and then generalizes image compression to video compression, by adding an interpolation loop into both encoder and decoder sides, to achieve higher image compression performance.

Object based Hybrid Video Compression

TLDR
This paper proposes replacing pixels with objects as the unit of compression while still harnessing the advantages of standard video codecs thereby reducing the bits required to represent a video scene while still achieving suitable visual quality in compressed videos.

Neural Inter-Frame Compression for Video Coding

TLDR
This work presents an inter-frame compression approach for neural video coding that can seamlessly build up on different existing neural image codecs and proposes to compute residuals directly in latent space instead of in pixel space to reuse the same image compression network for both key frames and intermediate frames.

Efficient Video Compression via Content-Adaptive Super-Resolution

TLDR
This paper presents a new approach that augments existing codecs with a small, content-adaptive super-resolution model that significantly boosts video quality and results show that to achieve the same PSNR, SRVC requires 20% of the bits-per-pixel of H.265 in slow mode, and 3% of that of DVC, a recent deep learning-based video compression scheme.

DVC: An End-To-End Deep Video Compression Framework

TLDR
This paper proposes the first end-to-end video compression deep model that jointly optimizes all the components for video compression, and shows that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard MS-SSIM.

Learned image and video compression with deep neural networks

TLDR
This tutorial will cover both the traditional data coding techniques and the popular learning based visual data compression algorithms, which will help the audiences with different backgrounds learn the recent progresses in this emerging research area.

Leveraging Bitstream Metadata for Fast and Accurate Video Compression Correction

TLDR
This work develops a deep learning architecture capable of restoring detail to compressed videos which leverages the underlying structure and motion information embedded in the video bitstream and shows that this improves restoration accuracy compared to prior compression correction methods and is competitive when compared with recent deep-learning-based video compression methods on rate-distortion.

Deep Generative Video Compression

TLDR
This work proposes an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video that builds upon variational autoencoder models for sequential data and combines them with recent work on neural image compression.

Deep Generative Video Compression

TLDR
This work proposes an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video that builds upon variational autoencoder models for sequential data and combines them with recent work on neural image compression.

Deep Generative Video Compression

TLDR
This work proposes an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video that builds upon variational autoencoder models for sequential data and combines them with recent work on neural image compression.
...

References

SHOWING 1-10 OF 33 REFERENCES

MPEG: a video compression standard for multimedia applications

TLDR
Design of the MPEG algorithm presents a difficult challenge since quality requirements demand high compression that cannot be achieved with only intraframe coding, and the algorithm’s random access requirement is best satisfied with pure intraframes coding.

Real-Time Adaptive Image Compression

TLDR
A machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time, and supplementing the approach with adversarial training specialized towards use in a compression setting.

Lossy Image Compression with Compressive Autoencoders

TLDR
It is shown that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs, and furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images.

Compressed Video Action Recognition

TLDR
This work proposes to train a deep network directly on the compressed video, using H.264, HEVC, etc., which has a higher information density, and found the training to be easier.

Learning to Inpaint for Image Compression

TLDR
Predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and learning to inpaint before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation.

Video Frame Synthesis Using Deep Voxel Flow

TLDR
This work addresses the problem of synthesizing new video frames in an existing video, either in-between existing frames (interpolation), or subsequent to them (extrapolation), by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which is called deep voxel flow.

Learning Binary Residual Representations for Domain-specific Video Streaming

TLDR
This work proposes a novel video compression pipeline that first applies H.264 to compress domain-specific videos, then trains a novel binary autoencoder to encode the leftover domain- specific residual information frame-by-frame into binary representations.

Video Codec Design: Developing Image and Video Compression Systems

TLDR
Video Codec Design sets out to de-mystify the subject of video coding and present a practical, design-based approach to this emerging field, providing communications engineers, system designers, researchers and technical managers with an essential handbook to image and video compression technology.

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

TLDR
This work proposes an end-to-end convolutional neural network for variable-length multi-frame video interpolation, where the motion interpretation and occlusion reasoning are jointly modeled.

Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks

TLDR
A method for lossy image compression based on recurrent, convolutional neural networks that outperforms BPG, WebP, JPEG2000, and JPEG as measured by MS-SSIM is proposed and it is shown that training with a pixel-wise loss weighted by SSIM increases reconstruction quality according to multiple metrics.