ClearShot: Eavesdropping on Keyboard Input from Video

@article{Balzarotti2008ClearShotEO,
  title={ClearShot: Eavesdropping on Keyboard Input from Video},
  author={Davide Balzarotti and Marco Cova and Giovanni Vigna},
  journal={2008 IEEE Symposium on Security and Privacy (sp 2008)},
  year={2008},
  pages={170-183}
}
Eavesdropping on electronic communication is usually prevented by using cryptography-based mechanisms. However, these mechanisms do not prevent one from obtaining private information through side channels, such as the electromagnetic emissions of monitors or the sound produced by keyboards. While extracting the same information by watching somebody typing on a keyboard might seem to be an easy task, it becomes extremely challenging if it has to be automated. However, an automated tool is needed… 

Figures and Tables from this paper

On the Privacy Risks of Virtual Keyboards: Automatic Reconstruction of Typed Input from Compromising Reflections

It is shown that so-called compromising reflections of a device's screen are sufficient to enable automated reconstruction, from video, of text typed on a virtual keyboard, and the importance of adjusting privacy expectations in response to emerging technologies is highlighted.

Beware, Your Hands Reveal Your Secrets!

A new breed of side-channel attack on the PIN entry process on a smartphone which entirely relies on the spatio-temporal dynamics of the hands during typing to decode the typed text and is very likely to be adopted by adversaries who seek to stealthily steal sensitive private information.

A fast eavesdropping attack against touchscreens

This paper presents an automatic attack against modern touchscreen keyboards that detects, tracks, and rectifies the target touchscreen, thus following the device or camera's movements and eliminating possible perspective distortions and rotations.

Don ’ t touch a word ! A practical input eavesdropping attack against mobile touchscreen devices

A practical eavesdropping attack, able to recognize the sequence of keystrokes from a low-resolution video, recorded while the victim is typing on a touchscreen, confirming that touchscreen keyboards that magnify keys make automatic eavesdropping attacks easier than in classic mobile keyboards.

Skype & Type: Keyboard Eavesdropping in Voice-over-IP

A new keyboard acoustic eavesdropping attack that involves VoIP, called Skype & Type (S&T), which demonstrates that, given some knowledge on the victim’s typing style and keyboard model, the attacker attains top-5 accuracy of 91.7% in guessing a random key pressed by the victim.

iSpy: automatic reconstruction of typed input from compromising reflections

The implications of the ubiquity of personal mobile devices are investigated and new techniques for compromising the privacy of users typing on virtual keyboards are revealed, highlighting the importance of adjusting privacy expectations in response to emerging technologies.

Stealing Passwords by Observing Hands Movement

Threats to smartphone users’ conventional login in public places are reaffirmed and the threats in scenarios such as hiding the screen that otherwise gives the impression of being safe to the users are highlighted.

WritingHacker: audio based eavesdropping of handwriting via mobile devices

This paper explores the possibility of eavesdropping on handwriting via nearby mobile devices based on audio signal processing and machine learning, and presents a proof-of-concept system, WritingHacker, which shows the usage of mobile devices to collect the sound of victims' handwriting, and to extract handwriting-specific features for machine learning based analysis.

Seeing double: reconstructing obscured typed input from repeated compromising reflections

This paper extends the scope of the attacks by relaxing requirements and shows that breaches of privacy are possible even when the adversary is around a corner and overcomes challenges posed by low image resolution by extending computer vision methods to operate on small, high-noise, images.

Keyboard acoustic side channel attacks: exploring realistic and security-sensitive scenarios

This work investigates several existing signal processing techniques for keyboard acoustic attack and introduces a novel technique—time–frequency decoding—that improves the detection accuracy compared to previous techniques.
...

References

SHOWING 1-10 OF 57 REFERENCES

Keyboards and Covert Channels

The experiments suggest that simple Keyboard JitterBugs can be a practical technique for capturing and exfiltrating typed secrets under conventional OSes and interactive network applications, even when the receiver is many hops away on the Internet.

Keyboard acoustic emanations revisited

An attack taking as input a 10-minute sound recording of a user typing English text using a keyboard and recovering up to 96% of typed characters is presented, using the statistical constraints of the underlying content, English language, to reconstruct text from sound recordings without knowing the corresponding clear text.

Compromising Reflections-or-How to Read LCD Monitors around the Corner

This work presents a novel eavesdropping technique that exploits reflections of the screen's optical emanations in various objects that one commonly finds in close proximity to the screen and uses those reflections to recover the original screen content.

Dictionary attacks using keyboard acoustic emanations

We present a dictionary attack that is based on keyboard acoustic emanations. We combine signal processing and efficient data structures and algorithms, to successfully reconstruct single words of

VISUAL PANEL: From an ordinary paper to a wireless and mobile input device

This paper presents a vision-based gesture interface system, VISUAL PANEL, which employs an arbitrary quadrangle-shaped panel and a tip pointer as an intuitive input device that can accurately and reliably track the panel and the tip pointer.

Optical time-domain eavesdropping risks of CRT displays

  • M. Kuhn
  • Physics
    Proceedings 2002 IEEE Symposium on Security and Privacy
  • 2002
Experiments show that enough high-frequency content remains in the emitted light to permit the reconstruction of readable text by deconvolving the signal received with a fast photosensor, and that optical compromising emanations can be received even after diffuse reflection from a wall.

Electromagnetic Eavesdropping Risks of Flat-Panel Displays

  • M. Kuhn
  • Computer Science
    Privacy Enhancing Technologies
  • 2004
The serial Gbit/s transmission formats used by modern digital video interfaces in effect modulate the signal, thereby making it even better suited for remote reception than emanations from analog systems.

Soft Tempest: Hidden Data Transmission Using Electromagnetic Emanations

Techniques that enable the software on a computer to control the electromagnetic radiation it transmits and a trusted screen driver can display sensitive information using fonts which minimise the energy of these emissions are discussed.

Language Identification of Encrypted VoIP Traffic: Alejandra y Roberto or Alice and Bob?

It is demonstrated that current cryptographic techniques do not provide adequate protection when the underlying audio is encoded using bandwidth-saving Variable Bit Rate (VBR) coders.

Electromagnetic radiation from video display units: An eavesdropping risk?

...