Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding

  title={Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding},
  author={Luka Murn and Saverio G. Blasi and Alan F. Smeaton and Marta Mrak},
The versatility of recent machine learning approaches makes them ideal for improvement of next generation video compression solutions. Unfortunately, these approaches typically bring significant increases in computational complexity and are difficult to interpret into explainable models, affecting their potential for implementation within practical video coding applications. This paper introduces a novel explainable neural network-based inter-prediction scheme, to improve the interpolation of… Expand


Interpreting CNN For Low Complexity Learned Sub-Pixel Motion Compensation In Video Coding
A novel neural network-based tool is presented which improves the interpolation of reference samples needed for fractional precision motion compensation by focusing on complexity reduction achieved by interpreting the interpolations filters learned by the networks. Expand
Attention-Based Neural Networks for Chroma Intra Prediction in Video Coding
A novel size-agnostic multi-model approach is proposed to reduce the complexity of the inference process, and the resulting simplified architecture is still capable of outperforming state-of-the-art methods. Expand
A convolutional neural network approach for half-pel interpolation in video coding
This work proposes to design a CNN-based interpolation filter (CNNIF) for video coding, Inspired by the great success of convolutional neural network (CNN) in computer vision, which achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration. Expand
Deep Learning-Based Luma and Chroma Fractional Interpolation in Video Coding
Inspired by the success of Convolutional Neural Network (CNN) in super-resolution, this work proposes CNN-based fractional interpolation for Luminance (Luma) and Chrominance (Chroma) components in motion compensated prediction to improve the coding efficiency. Expand
One-for-All: Grouped Variation Network-Based Fractional Interpolation in Video Coding
This paper presents a one-for-all fractional interpolation method based on a grouped variation convolutional neural network (GVCNN), which can deal with video frames coded using different QPs and is capable of generating all sub- pixel positions at one sub-pixel level. Expand
Enhanced Bi-Prediction With Convolutional Neural Network for High-Efficiency Video Coding
  • Zhenghui Zhao, Shiqi Wang, Shanshe Wang, Xinfeng Zhang, Siwei Ma, Jiansheng Yang
  • Computer Science
  • IEEE Transactions on Circuits and Systems for Video Technology
  • 2019
An enhanced bi-prediction scheme based on the convolutional neural network (CNN) to improve the rate-distortion performance in video compression by employing CNN to directly infer the predictive signals in a data-driven manner. Expand
Convolutional Neural Network-Based Motion Compensation Refinement for Video Coding
This work studies a simple CNN-based motion compensation refinement (CNNMCR) scheme, and proposes a more powerful CNNMCR scheme, where the CNN utilizes not only the motion compensated prediction, but also the neighboring reconstructed region to refine the prediction. Expand
A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
A CNN-based post-processing algorithm for High Efficiency Video Coding (HEVC), the state-of-the-art video coding standard, that outperforms previously studied networks in achieving higher bit-rate reduction, lower memory cost, and multiplied computational speedup. Expand
Convolutional Neural Network-Based Fractional-Pixel Motion Compensation
This paper forms the fractional-pixel reference generation CNN (FRCNN) as an inter-picture regression problem, which is to predict the pixel values of the current to-be-coded picture from the integer-pixel values of a reference picture, given a fractional -pixel motion vector that relates the two pictures. Expand
BVI-DVC: A Training Database for Deep Video Compression
A new extensive and representative video database, BVI-DVC, is presented for training CNN-based video compression systems, with specific emphasis on machine learning tools that enhance conventional coding architectures, including spatial resolution and bit depth up-sampling, post-processing and in-loop filtering. Expand