ClearShot: Eavesdropping on Keyboard Input from Video

@article{Balzarotti2008ClearShotEO,
  title={ClearShot: Eavesdropping on Keyboard Input from Video},
  author={Davide Balzarotti and Marco Cova and Giovanni Vigna},
  journal={2008 IEEE Symposium on Security and Privacy (sp 2008)},
  year={2008},
  pages={170-183}
}
Eavesdropping on electronic communication is usually prevented by using cryptography-based mechanisms. However, these mechanisms do not prevent one from obtaining private information through side channels, such as the electromagnetic emissions of monitors or the sound produced by keyboards. While extracting the same information by watching somebody typing on a keyboard might seem to be an easy task, it becomes extremely challenging if it has to be automated. However, an automated tool is needed… 
On the Privacy Risks of Virtual Keyboards: Automatic Reconstruction of Typed Input from Compromising Reflections
TLDR
It is shown that so-called compromising reflections of a device's screen are sufficient to enable automated reconstruction, from video, of text typed on a virtual keyboard, and the importance of adjusting privacy expectations in response to emerging technologies is highlighted.
Beware, Your Hands Reveal Your Secrets!
TLDR
A new breed of side-channel attack on the PIN entry process on a smartphone which entirely relies on the spatio-temporal dynamics of the hands during typing to decode the typed text and is very likely to be adopted by adversaries who seek to stealthily steal sensitive private information.
A fast eavesdropping attack against touchscreens
TLDR
This paper presents an automatic attack against modern touchscreen keyboards that detects, tracks, and rectifies the target touchscreen, thus following the device or camera's movements and eliminating possible perspective distortions and rotations.
Don ’ t touch a word ! A practical input eavesdropping attack against mobile touchscreen devices
Spying on a person is a subtle, yet easy and reliable method to obtain sensitive information. Even if the victim is well protected from digital attacks, spying may be a viable option. In addition,
Skype & Type: Keyboard Eavesdropping in Voice-over-IP
Voice-over-IP (VoIP) software are among the most widely spread and pervasive software, counting millions of monthly users. However, we argue that people ignore the drawbacks of transmitting
iSpy: automatic reconstruction of typed input from compromising reflections
TLDR
The implications of the ubiquity of personal mobile devices are investigated and new techniques for compromising the privacy of users typing on virtual keyboards are revealed, highlighting the importance of adjusting privacy expectations in response to emerging technologies.
Stealing Passwords by Observing Hands Movement
TLDR
Threats to smartphone users’ conventional login in public places are reaffirmed and the threats in scenarios such as hiding the screen that otherwise gives the impression of being safe to the users are highlighted.
WritingHacker: audio based eavesdropping of handwriting via mobile devices
TLDR
This paper explores the possibility of eavesdropping on handwriting via nearby mobile devices based on audio signal processing and machine learning, and presents a proof-of-concept system, WritingHacker, which shows the usage of mobile devices to collect the sound of victims' handwriting, and to extract handwriting-specific features for machine learning based analysis.
Seeing double: reconstructing obscured typed input from repeated compromising reflections
TLDR
This paper extends the scope of the attacks by relaxing requirements and shows that breaches of privacy are possible even when the adversary is around a corner and overcomes challenges posed by low image resolution by extending computer vision methods to operate on small, high-noise, images.
Keyboard acoustic side channel attacks: exploring realistic and security-sensitive scenarios
TLDR
This work investigates several existing signal processing techniques for keyboard acoustic attack and introduces a novel technique—time–frequency decoding—that improves the detection accuracy compared to previous techniques.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 59 REFERENCES
Keyboards and Covert Channels
TLDR
The experiments suggest that simple Keyboard JitterBugs can be a practical technique for capturing and exfiltrating typed secrets under conventional OSes and interactive network applications, even when the receiver is many hops away on the Internet.
Compromising Reflections-or-How to Read LCD Monitors around the Corner
TLDR
This work presents a novel eavesdropping technique that exploits reflections of the screen's optical emanations in various objects that one commonly finds in close proximity to the screen and uses those reflections to recover the original screen content.
Keyboard acoustic emanations revisited
TLDR
A novel attack taking as input a 10-minute sound recording of a user typing English text using a keyboard, and then recovering up to 96% of typed characters, without any need for a labeled training recording is presented.
Dictionary attacks using keyboard acoustic emanations
We present a dictionary attack that is based on keyboard acoustic emanations. We combine signal processing and efficient data structures and algorithms, to successfully reconstruct single words of
Timing Analysis of Keystrokes and Timing Attacks on SSH
TLDR
A statistical study of users' typing patterns is performed and it is shown that these patterns reveal information about the keys typed, and that timing leaks open a new set of security risks, and hence caution must be taken when designing this type of protocol.
VISUAL PANEL: From an ordinary paper to a wireless and mobile input device
TLDR
This paper presents a vision-based gesture interface system, VISUAL PANEL, which employs an arbitrary quadrangle-shaped panel and a tip pointer as an intuitive input device that can accurately and reliably track the panel and the tip pointer.
Optical time-domain eavesdropping risks of CRT displays
  • M. Kuhn
  • Computer Science
    Proceedings 2002 IEEE Symposium on Security and Privacy
  • 2002
TLDR
Experiments show that enough high-frequency content remains in the emitted light to permit the reconstruction of readable text by deconvolving the signal received with a fast photosensor, and that optical compromising emanations can be received even after diffuse reflection from a wall.
Electromagnetic Eavesdropping Risks of Flat-Panel Displays
  • M. Kuhn
  • Computer Science
    Privacy Enhancing Technologies
  • 2004
TLDR
The serial Gbit/s transmission formats used by modern digital video interfaces in effect modulate the signal, thereby making it even better suited for remote reception than emanations from analog systems.
Soft Tempest: Hidden Data Transmission Using Electromagnetic Emanations
TLDR
Techniques that enable the software on a computer to control the electromagnetic radiation it transmits and a trusted screen driver can display sensitive information using fonts which minimise the energy of these emissions are discussed.
Language Identification of Encrypted VoIP Traffic: Alejandra y Roberto or Alice and Bob?
TLDR
It is demonstrated that current cryptographic techniques do not provide adequate protection when the underlying audio is encoded using bandwidth-saving Variable Bit Rate (VBR) coders.
...
1
2
3
4
5
...