Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency

@article{Chiang2020DeployingID,
  title={Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency},
  author={Cheng-Ming Chiang and Yu Tseng and Yu-Syuan Xu and Hsien-Kai Kuo and Yi-Min Tsai and G. Chen and Koan-Sin Tan and Wei-Ting Wang and yu-chieh lin and Shou-Yao Roy Tseng and Wei-Shiang Lin and Chia-Lin Yu and B-Y. Shen and Kloze Kao and Chia-Ming Cheng and Hung-Jen Chen},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  year={2020},
  pages={2109-2119}
}
Recently, image enhancement and restoration have become important applications on mobile devices, such as super-resolution and image deblurring. However, most state-of-the-art networks present extremely high computational complexity. This makes them difficult to be deployed on mobile devices with acceptable latency. Moreover, when deploying to different mobile devices, there is a large latency variation due to the difference and limitation of deep learning accelerators on mobile devices. In… Expand
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
  • Andrey D. Ignatov, Kim Byeoung-su, +29 authors Feifei Chen
  • Computer Science, Engineering
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2021
TLDR
The first Mobile AI challenge is introduced, where the target is to develop an end-to-end deep learning-based image denoising solution that can demonstrate high efficiency on smartphone GPUs. Expand
Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
TLDR
The target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a realtime performance on mobile GPUs and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results. Expand
Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report
  • Andrey D. Ignatov, R. Timofte, +20 authors Shengpeng Wang
  • Computer Science, Engineering
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2021
TLDR
The first Mobile AI challenge is introduced, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a real-time performance on mobile or edge NPUs. Expand
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
TLDR
This paper introduces the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based depth estimation solutions that can demonstrate a nearly realtime performance on smartphones and IoT platforms. Expand
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
TLDR
The first Mobile AI challenge is introduced, where the target is to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms. Expand
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
TLDR
The target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. Expand
NTIRE 2020 Challenge on Image and Video Deblurring
TLDR
This challenge presents the evaluation results from 3 competition tracks as well as the proposed solutions, and the winning methods demonstrate the state-of-the-art performance on image and video deblurring tasks. Expand
Fast and Accurate Camera Scene Detection on Smartphones
TLDR
A novel Camera Scene Detection Dataset (CamSDD) containing more than 11K manually crawled images belonging to 30 different scene categories is proposed and an efficient and NPU-friendly CNN model is proposed that demonstrates a top-3 accuracy of 99.5% on this dataset and achieves more than 200 FPS on the recent mobile SoCs. Expand
Binarized Weight Error Networks With a Transition Regularization Term
This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account theExpand

References

SHOWING 1-10 OF 45 REFERENCES
PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report
TLDR
This paper reviews the first challenge on efficient perceptual image enhancement with the focus on deploying deep learning models on smartphones and proposes solutions that significantly improved baseline results defining the state-of-the-art for image enhancement on smartphones. Expand
Region-Adaptive Dense Network for Efficient Motion Deblurring
TLDR
This paper presents a new architecture composed of region adaptive dense deformable modules that implicitly discover the spatially varying shifts responsible for non-uniform blur in the input image and learn to modulate the filters, enabling almost real-time deblurring. Expand
Replacing Mobile Camera ISP with a Single Deep Learning Model
TLDR
PyNET is presented, a novel pyramidal CNN architecture designed for fine-grained image restoration that implicitly learns to perform all ISP steps such as image demosaicing, denoising, white balancing, color and contrast correction, demoireing, etc. Expand
NTIRE 2020 Challenge on Image and Video Deblurring
TLDR
This challenge presents the evaluation results from 3 competition tracks as well as the proposed solutions, and the winning methods demonstrate the state-of-the-art performance on image and video deblurring tasks. Expand
Spatially-Adaptive Residual Networks for Efficient Image and Video Deblurring
TLDR
Extensive qualitative and quantitative comparisons with prior art on benchmark dynamic scene deblurring datasets clearly demonstrate the superiority of the proposed networks via a reduction in model-size and significant improvements in accuracy and speed, enabling almost real-timedeblurring. Expand
AI Benchmark: All About Deep Learning on Smartphones in 2019
TLDR
This paper evaluates the performance and compares the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference and discusses the recent changes in the Android ML pipeline. Expand
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
TLDR
It is demonstrated that DeblurGAN-V2 has very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency, and is effective for general image restoration tasks too. Expand
Mobile-Deblur: A Clear Image Will on the Smart Device
TLDR
This paper proposed portable deblurring system for mobile terminal of fuzzy pictures based on two classic image restoration algorithm of Wiener filter and blind restoration, which adaptively evaluates the degree of picture blurs and the point diffusion function is resolved by the existing fuzzy picture feature. Expand
Efficient Super Resolution Using Binarized Neural Network
TLDR
This work explores an network-binarization approach for SR tasks without sacrificing much reconstruction accuracy, and shows that binarized SR networks achieve comparable qualitative and quantitative results as their real-weight counterparts. Expand
Learning to See in the Dark
TLDR
A pipeline for processing low-light images is developed, based on end-to-end training of a fully-convolutional network that operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. Expand
...
1
2
3
4
5
...