Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers

@article{Biener2020BreakingTS,
  title={Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers},
  author={Verena Biener and Danielle Eve Schneider and Travis Gesslein and Alexander Otte and Bastian Kuth and Per Ola Kristensson and Eyal Ofek and Michel Pahud and Jens Grubert},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2020},
  volume={26},
  pages={3490-3502}
}
Virtual Reality (VR) has the potential to transform knowledge work. One advantage of VR knowledge work is that it allows extending 2D displays into the third dimension, enabling new operations, such as selecting overlapping objects or displaying additional layers of information. On the other hand, mobile knowledge workers often work on established mobile devices, such as tablets, limiting interaction with those devices to a small input space. This challenge of a constrained input space is… Expand
Pen-based Interaction with Spreadsheets in Mobile Virtual Reality
TLDR
A tool-set for enhancing spreadsheet interaction on tablets using immersive VR headsets and pen-based input is presented and tools for the efficient creation and editing of spreadsheets functions such as off-the-screen layered menus, visualization of sheets dependencies, and gaze-and-touch-based switching between spreadsheet tabs are proposed. Expand
Extended Reality for Knowledge Work in Everyday Environments
TLDR
This chapter summarizes the state of the art in the field of extended reality for knowledge work in everyday environments and proposes steps to address the open challenges. Expand
Towards a Practical Virtual Office for Mobile Knowledge Workers
TLDR
This paper investigates opportunities and challenges for realizing a mobile VR offices environments and discusses implications from recent findings of mixing standard off-the-shelf equipment, such as tablets, laptops or desktops, with VR to enable effective, efficient, ergonomic, and rewarding mobile knowledge work. Expand
STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics
TLDR
A novel eyes-free interaction concept for the seamless transition between the tablet and the augmented reality environment is developed and a user study reveals that participants appreciated the novel interaction concept, indicating the potential for spatially-aware tablets in augmented reality. Expand
Phonetroller: Visual Representations of Fingers for Precise Touch Input with Mobile Phones in VR
TLDR
The results show that the enabled style of feedback is effective for thumb-size targets, and that the VR experience can be enriched by using smartphones as VR controllers supporting precise touch input. Expand
The MADE-Axis
  • J. Smiley, Benjamin Lee, +5 authors T. Dwyer
  • Computer Science
  • Proc. ACM Hum. Comput. Interact.
  • 2021
TLDR
This work decomposes a bespoke controller for immersive data visualisation into a simple, wireless, composable unit featuring two actuated sliders and a rotary encoder, showing that the flexibility and composability of these actuated controllers provides an emergent design space which covers the range of interactive dynamics for visual analysis. Expand
Do we still need physical monitors? An evaluation of the usability of AR virtual monitors for productivity work
TLDR
Results show that virtual monitors are a feasible approach for performing serious productivity work, albeit currently constrained by technical limitations that lead to inferior usability and performance compared to physical monitors. Expand
Mixed Reality Interaction Techniques
TLDR
This chapter gives an overview of interaction techniques for mixed reality including augmented and virtual reality (AR/VR) and the combination of multiple modalities in multisensory and multimodal interaction. Expand
A Reconfigurable Mobile Head-Mounted Display Supporting Real World Interactions
We propose a new mobile head-mounted display, ModularHMD, that uses a modular mechanism with a manually reconfigurable structure to enable ad-hoc peripheral interaction with real-world objects andExpand
ModularHMD: A Reconfigurable Mobile Head-Mounted Display Enabling Ad-hoc Peripheral Interactions with the Real World
We propose ModularHMD, a new mobile head-mounted display concept, which adopts a modular mechanism and allows a user to perform ad-hoc peripheral interaction with real-world devices or people duringExpand
...
1
2
...

References

SHOWING 1-10 OF 143 REFERENCES
Pen-based Interaction with Spreadsheets in Mobile Virtual Reality
TLDR
A tool-set for enhancing spreadsheet interaction on tablets using immersive VR headsets and pen-based input is presented and tools for the efficient creation and editing of spreadsheets functions such as off-the-screen layered menus, visualization of sheets dependencies, and gaze-and-touch-based switching between spreadsheet tabs are proposed. Expand
TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality
TLDR
Observations gained during semi-structured interviews with general users, and those experienced with 3D software, are used to develop a vocabulary of interaction techniques to demonstrate how a tablet's precise touch input capability, physical shape, metaphorical associations, and natural compatibility with barehand mid-air input can be used in VR. Expand
A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays
TLDR
Usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) are identified in a survey of 108 VR HMD users, and it is shown that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. Expand
"In VR, everything is possible!": Sketching and Simulating Spatially-Aware Interactive Spaces in Virtual Reality
TLDR
Using virtual reality (VR) as a design tool for sketching and simulating spatially-aware interactive spaces and playful design explorations as key opportunities is proposed. Expand
Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
TLDR
It is concluded that optimizing the visualization of hands in VR is important, especially for inexperienced typists, to enable a high typing performance. Expand
Towards a Practical Virtual Office for Mobile Knowledge Workers
TLDR
This paper investigates opportunities and challenges for realizing a mobile VR offices environments and discusses implications from recent findings of mixing standard off-the-shelf equipment, such as tablets, laptops or desktops, with VR to enable effective, efficient, ergonomic, and rewarding mobile knowledge work. Expand
FaceTouch: Enabling Touch Interaction in Display Fixed UIs for Mobile Virtual Reality
TLDR
FaceTouch, a novel interaction concept for mobile Virtual Reality (VR) head-mounted displays (HMDs) that leverages the backside as a touch-sensitive surface, is presented and interaction techniques and three example applications that explore the FaceTouch design space are presented. Expand
Gaze + pinch interaction in virtual reality
TLDR
Novel interaction concepts and an experimental system prototype that bring together interaction technique variants, menu interfaces, and applications into one unified virtual experience are described. Expand
VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality
TLDR
VRSketchIn is presented, a VR sketching application using a 6DoF-tracked pen and a 6 doF-Tracked tablet as input devices, combining unconstrained 3D mid-air with constrained 2D surface-based sketching. Expand
User evaluations on form factors of tangible magic lenses
  • Ji-Young Oh, H. Hua
  • Computer Science
  • 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality
  • 2006
TLDR
This paper conducted two sets of user evaluations, one mainly testing the lens aspect ratio, and another for the lens size, and found that the aspect ratio of a lens plays more important role in user preference for smaller lenses than for larger ones. Expand
...
1
2
3
4
5
...