TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors

@inproceedings{Xu2012TapLoggerIU,
  title={TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors},
  author={Zhi Xu and Kun Bai and Sencun Zhu},
  booktitle={Wireless Network Security},
  year={2012}
}
Today's smartphones are shipped with various embedded motion sensors, such as the accelerometer, gyroscope, and orientation sensors. [] Key Method Specifically, we utilize an installed trojan application to stealthily monitor the movement and gesture changes of a smartphone using its on-board motion sensors. When the user is interacting with the trojan application, it learns the motion change patterns of tap events.

TextLogger: inferring longer inputs on touch screen using motion sensors

The feasibility of inferring long user inputs to readable sentences from motion sensor data is shown, which shows that more sensitive information about the device owners can be exposed by applying text mining technology on the inferred text.

On motion sensors as source for user input inference in smartphones

It is concluded that readings from motion sensor are a powerful side channel for inferring user inputs, and could provide extra avenues for attackers.

Input extraction via motion-sensor behavior analysis on smartphones

Inferring Touch from Motion in Real World Data

This paper presents a side channel attack on touch input by analyzing motion sensor readings and uses a classifier based on the Dynamic Time Warping algorithm to infer touch from motion inputs.

Using Unrestricted Mobile Sensors to Infer Tapped and Traced User Inputs

  • Trang Nguyen
  • Computer Science
    2015 12th International Conference on Information Technology - New Generations
  • 2015
This work demonstrates that it is indeed possible to recover both tap and trace inputted text using only motion sensor data and develops an application that can use the gyroscope and accelerometer to interpret what the user has written, even if trace input is used.

You Are How You Touch: User Verification on Smartphones via Tapping Behaviors

This work proposes a non-intrusive user verification mechanism to substantiate whether an authenticating user is the true owner of the smart phone or an impostor who happens to know the pass code.

I Know What You Type on Your Phone: Keystroke Inference on Android Device Using Deep Learning

A deep neural network with four hidden layers is proposed as the baseline for this work, which achieves an accuracy of 47% using categorical cross entropy as the accuracy metric.

PIN skimmer: inferring PINs through the camera and microphone

It turns out to be difficult to prevent such side-channel attacks, so guidelines for developers to mitigate present and future side- channel attacks on PIN input are provided.

Slogger: Smashing Motion-based Touchstroke Logging with Transparent System Noise

A novel and practical defense to motion-based touchstroke leakage based on system-generated, fully automated and user-oblivious sensory noise, and it is shown that the leakage can be minimized even when attacks utilize a fusion of multiple motion-position sensors.

Age group detection using smartphone motion sensors

This paper describes a new side-channel attack on smartphones that aims to predict the age interval of the user and develops an Android application that evaluates accelerometer sensor data and performs child/adult detection with a success rate of 92.5%.
...

References

SHOWING 1-10 OF 40 REFERENCES

(sp)iPhone: decoding vibrations from nearby keyboards using mobile phone accelerometers

It is demonstrated that an application with access to accelerometer readings on a modern mobile phone can use such information to recover text entered on a nearby keyboard, and the potential to recover significant information from the vicinity of a mobile device without gaining access to resources generally considered to be the most likely sources of leakage.

TouchLogger: Inferring Keystrokes on Touch Screen from Smartphone Motion

This work describes a new side channel, motion, on touch screen smartphones with only soft keyboards, and developed TouchLogger, an Android application that extracts features from device orientation data to infer keystrokes.

ACCessory: password inference using accelerometers on smartphones

It is shown that accelerometer measurements can be used to extract 6-character passwords in as few as 4.5 trials (median) and unlike many other sensors found on smartphones, the accelerometer does not require special privileges to access on current smartphone OSes.

Smudge Attacks on Smartphone Touch Screens

This paper examines the feasibility of smudge attacks on touch screens for smartphones, and focuses on the Android password pattern, and provides a preliminary analysis of applying the information learned in a smudge attack to guessing an Android passwordpattern.

A survey of mobile phone sensing

This article surveys existing mobile phone sensing algorithms, applications, and systems, and discusses the emerging sensing paradigms, and formulates an architectural framework for discussing a number of the open issues and challenges emerging in the new area ofMobile phone sensing research.

SenSay: a context-aware mobile phone

Results from the threshold analyses show a clear delineation can be made among several user states by examining sensor data trends.

ACComplice: Location inference using accelerometers on smartphones

It is demonstrated that accelerometers can be used to locate a device owner to within a 200 meter radius of the true location and are comparable to the typical accuracy for handheld global positioning systems.

Poster: fast, automatic iPhone shoulder surfing

This work proposes an automatic shoulder surfing attack against modern touchscreen keyboards that display magnified keys in predictable positions that requires no particular settings and even allows for natural movements of both target device and shoulder surfer's camera, and accurately recovers the sequence of keystrokes input by the user.

Taming Information-Stealing Smartphone Applications (on Android)

A system called TISSA is developed that implements a new privacy mode in smartphones that can empower users to flexibly control in a fine-grained manner what kinds of personal information will be accessible to an application.

iSpy: automatic reconstruction of typed input from compromising reflections

The implications of the ubiquity of personal mobile devices are investigated and new techniques for compromising the privacy of users typing on virtual keyboards are revealed, highlighting the importance of adjusting privacy expectations in response to emerging technologies.