TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors

@inproceedings{Xu2012TapLoggerIU,
  title={TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors},
  author={Z. Xu and Kun Bai and Sencun Zhu},
  booktitle={WISEC '12},
  year={2012}
}
Today's smartphones are shipped with various embedded motion sensors, such as the accelerometer, gyroscope, and orientation sensors. [...] Key Method Specifically, we utilize an installed trojan application to stealthily monitor the movement and gesture changes of a smartphone using its on-board motion sensors. When the user is interacting with the trojan application, it learns the motion change patterns of tap events.Expand
View on ACM
cse.psu.edu
321 Citations
Highly Influential Citations
25
Background Citations
154
Methods Citations
62
Results Citations
10

321 Citations

Citation Type

Citation Type

TextLogger: inferring longer inputs on touch screen using motion sensors
TLDR
The feasibility of inferring long user inputs to readable sentences from motion sensor data is shown, which shows that more sensitive information about the device owners can be exposed by applying text mining technology on the inferred text. Expand
On motion sensors as source for user input inference in smartphones
TLDR
It is concluded that readings from motion sensor are a powerful side channel for inferring user inputs, and could provide extra avenues for attackers. Expand
Inferring Smartphone Users' Handwritten Patterns by using Motion Sensors
TLDR
This paper investigates the feasibility of inferring a user’s handwritten pattern on a smartphone touchscreen by using the embedded motion sensors and develops a novel sensor fusion mechanism to integrate information contained in multiple motion sensors by exploiting the majority voting strategy. Expand
Input extraction via motion-sensor behavior analysis on smartphones
TLDR
Readings from accelerometer and magnetometer data could be a powerful side channel for inferring user inputs on Android smartphones, and results indicated that users' inputs can be accurately inferred from the sensor data. Expand
WristSnoop: Smartphone PINs prediction using smartwatch motion sensors
TLDR
This study determines the feasibility and accuracy of inferring user keystrokes on a smartphone through a smartwatch worn by the user and shows that with malware accessing only the smartwatch's motion sensors, it is possible to recognize user activity and specific numeric keypad entries. Expand
Inferring Touch from Motion in Real World Data
TLDR
This paper presents a side channel attack on touch input by analyzing motion sensor readings and uses a classifier based on the Dynamic Time Warping algorithm to infer touch from motion inputs. Expand
Using Unrestricted Mobile Sensors to Infer Tapped and Traced User Inputs
  • Trang Nguyen
  • Computer Science
  • 2015 12th International Conference on Information Technology - New Generations
  • 2015
TLDR
This work demonstrates that it is indeed possible to recover both tap and trace inputted text using only motion sensor data and develops an application that can use the gyroscope and accelerometer to interpret what the user has written, even if trace input is used. Expand
Practicality of accelerometer side channels on smartphones
TLDR
This paper demonstrates how to use the accelerometer sensor to learn user tap- and gesture-based input as required to unlock smartphones using a PIN/password or Android's graphical password pattern and develops sample rate independent features for accelerometer readings based on signal processing and polynomial fitting techniques. Expand
You Are How You Touch: User Verification on Smartphones via Tapping Behaviors
TLDR
This work proposes a non-intrusive user verification mechanism to substantiate whether an authenticating user is the true owner of the smart phone or an impostor who happens to know the pass code. Expand
I Know What You Type on Your Phone: Keystroke Inference on Android Device Using Deep Learning
TLDR
A deep neural network with four hidden layers is proposed as the baseline for this work, which achieves an accuracy of 47% using categorical cross entropy as the accuracy metric. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 49 REFERENCES
(sp)iPhone: decoding vibrations from nearby keyboards using mobile phone accelerometers
TLDR
It is demonstrated that an application with access to accelerometer readings on a modern mobile phone can use such information to recover text entered on a nearby keyboard, and the potential to recover significant information from the vicinity of a mobile device without gaining access to resources generally considered to be the most likely sources of leakage. Expand
TouchLogger: Inferring Keystrokes on Touch Screen from Smartphone Motion
  • Liang Cai, Hao Chen
  • Computer Science
  • HotSec
  • 2011
TLDR
This work describes a new side channel, motion, on touch screen smartphones with only soft keyboards, and developed TouchLogger, an Android application that extracts features from device orientation data to infer keystrokes. Expand
ACCessory: password inference using accelerometers on smartphones
TLDR
It is shown that accelerometer measurements can be used to extract 6-character passwords in as few as 4.5 trials (median) and unlike many other sensors found on smartphones, the accelerometer does not require special privileges to access on current smartphone OSes. Expand
Soundcomber: A Stealthy and Context-Aware Sound Trojan for Smartphones
TLDR
This work presents Soundcomber, a Trojan with few and innocuous permissions, that can extract a small amount of targeted private information from the audio sensor of the phone, and performs efficient, stealthy local extraction, thereby greatly reducing the communication cost for delivering stolen data. Expand
Smudge Attacks on Smartphone Touch Screens
TLDR
This paper examines the feasibility of smudge attacks on touch screens for smartphones, and focuses on the Android password pattern, and provides a preliminary analysis of applying the information learned in a smudge attack to guessing an Android passwordpattern. Expand
A survey of mobile phone sensing
TLDR
This article surveys existing mobile phone sensing algorithms, applications, and systems, and discusses the emerging sensing paradigms, and formulates an architectural framework for discussing a number of the open issues and challenges emerging in the new area ofMobile phone sensing research. Expand
SenSay: a context-aware mobile phone
TLDR
Results from the threshold analyses show a clear delineation can be made among several user states by examining sensor data trends. Expand
ACComplice: Location inference using accelerometers on smartphones
TLDR
It is demonstrated that accelerometers can be used to locate a device owner to within a 200 meter radius of the true location and are comparable to the typical accuracy for handheld global positioning systems. Expand
Poster: fast, automatic iPhone shoulder surfing
TLDR
This work proposes an automatic shoulder surfing attack against modern touchscreen keyboards that display magnified keys in predictable positions that requires no particular settings and even allows for natural movements of both target device and shoulder surfer's camera, and accurately recovers the sequence of keystrokes input by the user. Expand
Taming Information-Stealing Smartphone Applications (on Android)
TLDR
A system called TISSA is developed that implements a new privacy mode in smartphones that can empower users to flexibly control in a fine-grained manner what kinds of personal information will be accessible to an application. Expand
...
1
2
3
4
5
...