• Corpus ID: 197638536

Content Assisted Viewport Prediction for Panorammic Video Streaming

@inproceedings{Xu2019ContentAV,
  title={Content Assisted Viewport Prediction for Panorammic Video Streaming},
  author={Tan Xu and T Labs and B. Han},
  year={2019}
}
In this paper, we explore the viewport prediction problem for 360-degree video streaming by utilizing a viewer’s recent head movement trajectory, cross-viewer heatmap, and video saliency detection. We propose a deep neural network (DNN) model using long short-term memory network (LSTM) as its backbone. This model fuses multi-modality features and makes a joint prediction for a user’s future viewing direction. We evaluate the proposed approach on a dataset recording the viewing sessions of more… 

Figures from this paper

Analyzing viewport prediction under different VR interactions
TLDR
This paper collects viewport trajectory traces from 275 users who have watched popular 360° panoramic videos for a total duration of 156 hours, and applies diverse machine learning algorithms - from simple regression to sophisticated deep learning that leverages crowd-sourced data - to analyze the performance of viewport prediction.
Analyzing Viewport Prediction Under Different VR Interactions
TLDR
This paper collects viewport trajectory traces from 275 users who have watched popular 360° panoramic videos for a total duration of 156 hours, and applies diverse machine learning algorithms – from simple regression to sophisticated deep learning that leverages crowd-sourced data – to analyze the performance of viewport prediction.
SalientVR: Saliency-Driven Mobile 360-Degree Video Streaming with Gaze Information
TLDR
This work designs a precise gaze-driven saliency judging criterion for mobile VR viewers, and designs a lightweight saliency-aware quality adaptation algorithm with a motion-assisted online correction, which is robust to wireless bandwidth vagaries and saliency bias.

References

SHOWING 1-5 OF 5 REFERENCES
Shooting a moving target: Motion-prediction-based transmission for 360-degree videos
TLDR
This paper proposes a machine learning mechanism that predicts not only viewer motion but also prediction deviation itself, and proposes a targeted transmission mechanism that minimizes overall bandwidth consumption while providing probabilistic performance guarantees.
Flare: Practical Viewport-Adaptive 360-Degree Video Streaming for Mobile Devices
TLDR
This work conducts an IRB-approved user study and develops novel online algorithms that determine which spatial portions to fetch and their corresponding qualities for Flare, a practical system for streaming 360-degree videos on commodity mobile devices.
Optimizing 360 video delivery over cellular networks
TLDR
This paper proposes a cellular-friendly streaming scheme that delivers only 360 videos' visible portion based on head movement prediction, which can reduce bandwidth consumption by up to 80% based on a trace-driven simulation.
A saliency-based search mechanism for overt and covert shifts of visual attention
Learning to learn by gradient descent by gradient descent
TLDR
This paper shows how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way.