How Good Are Humans at Solving CAPTCHAs? A Large Scale Evaluation

@article{Bursztein2010HowGA,
  title={How Good Are Humans at Solving CAPTCHAs? A Large Scale Evaluation},
  author={Elie Bursztein and Steven Bethard and Celine Fabry and John C. Mitchell and Dan Jurafsky},
  journal={2010 IEEE Symposium on Security and Privacy},
  year={2010},
  pages={399-413}
}
Captchas are designed to be easy for humans but hard for machines. However, most recent research has focused only on making them hard for machines. In this paper, we present what is to the best of our knowledge the first large scale evaluation of captchas from the human perspective, with the goal of assessing how much friction captchas present to the average user. For the purpose of this study we have asked workers from Amazon’s Mechanical Turk and an underground captchabreaking service to… Expand
View on IEEE
stanford.edu
247 Citations
Highly Influential Citations
25
Background Citations
109
Methods Citations
26
Results Citations
11

247 Citations

Citation Type

Citation Type

CAPTCHA Based on Human Cognitive Factor
TLDR
This paper addresses an alternative form of CAPTCHA that provides a variety of questions from mathematical, logical and general problems which only human can understand and answer correctly in a given time and shows the efficacy of the implemented system with a good level of user satisfaction. Expand
FR-CAPTCHA: CAPTCHA Based on Recognizing Human Faces
TLDR
This work proposes FR-CAPTCHA, a Turing test based on finding matching pairs of human faces in an image which achieves a human accuracy of 94% and is robust against automated attacks. Expand
I am Robot: (Deep) Learning to Break Semantic Image CAPTCHAs
TLDR
A comprehensive study of reCaptcha is conducted, and a novel low-cost attack that leverages deep learning technologies for the semantic annotation of images is designed, which is extremely effective and applies to the Facebook image captcha. Expand
言語的作問技法を用いたCAPTCHAの構築 = Design and implementation of CAPTCHA using verbal test
TLDR
A new CAPTCHA system which can differentiate between humans and software agents pretending to be humans, based on their different contextual cognition, is proposed, which works without relying on the specific perceptual abilities of the users. Expand
The Failure of Noise-Based Non-continuous Audio Captchas
TLDR
Decaptcha's performance on actual observed and synthetic CAPT CHAs indicates that such speech CAPTCHAs are inherently weak and, because of the importance of audio for various classes of users, alternative audio CAPTChAs must be developed. Expand
HuMan: an accessible, polymorphic and personalized CAPTCHA interface with preemption feature tailored for persons with visual impairments
TLDR
The HuMan model has a CAPTCHA preemption feature which enables the user to stop the challenge audio as soon as the answer is identified, and the polymorphic nature of resolving the HuManCAPTCHA facilitates kaleidoscopic behavior in CAPTcha rendering. Expand
Vulnerability of the conventional accessible CAPTCHA used by the White House and an alternative approach for visually impaired people
TLDR
It is demonstrated that the solver program can beat the CAPTCHA with a success rate of over 99% and a new verbal-style system to replace the quiz-based CAPTcha system is proposed, which is universally suitable for any type of perceptual channel. Expand
jCAPTCHA: Accessible Human Validation
CAPTCHAs are a widely deployed mechanism for ensuring that a web site user is a human, and not a software agent. They ought to be relatively easy for a human to solve, but hard for software to… Expand
An Accessible CAPTCHA System for People with Visual Disability - Generation of Human/Computer Distinguish Test with Documents on the Net
TLDR
A new scheme of CAPTCHA that does not become a perceptual barrier for disable people is proposed, designed to imitate the phenomenon called “consonant gradation” of natural languages. Expand
Blind and Human: Exploring More Usable Audio CAPTCHA Designs
TLDR
Four novel audio CAPTCHAs were designed and evaluated that were significantly more accurate and faster than the control condition, and were rated as preferable over the control. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 31 REFERENCES
Breaking Audio CAPTCHAs
TLDR
This work analyzed the security of current audio CAPTCHAs from popular Web sites by using AdaBoost, SVM, and k-NN, and achieved correct solutions for test samples with accuracy up to 71%. Expand
Asirra: a CAPTCHA that exploits interest-aligned manual image categorization
TLDR
A CAPTCHA that asks users to identify cats out of a set of 12 photographs of both cats and dogs, and two novel algorithms for amplifying the skill gap between humans and computers that can be used on many existing CAPTCHAs are described. Expand
Enhanced CAPTCHAs: Using Animation to Tell Humans and Computers Apart
TLDR
Animated CAPTCHAs are sealed against the Laundry attack by adding a dimension not used so far: animation, which ensures that unsuspected visitors will provide answers that will be useless on the attacker's side. Expand
Evaluating existing audio CAPTCHAs and an interface optimized for non-visual use
TLDR
A new interface for solving CAPTCHAs optimized for non-visual use that can be added in-place to existing audio CAPT CHAs is developed and evaluated, illustrating a broadly applicable principle of accessible design: the most usable audio interfaces are often not direct translations of existing visual interfaces. Expand
Recognizing objects in adversarial clutter: breaking a visual CAPTCHA
  • Greg Mori, Jitendra Malik
  • Computer Science
  • 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings.
  • 2003
TLDR
Efficient methods based on shape context matching are developed that can identify the word in an EZ-Gimpy image with a success rate of 92%, and the requisite 3 words in a Gimpy image 33% of the time. Expand
A low-cost attack on a Microsoft captcha
TLDR
It is shown that CAPTCHAs that are carefully designed to be segmentation-resistant are vulnerable to novel but simple attacks, including the schemes designed and deployed by Microsoft, Yahoo and Google. Expand
Computers beat Humans at Single Character Recognition in Reading based Human Interaction Proofs (HIPs)
TLDR
Comparisons of human and computer single character recognition abilities through a sequence of human user studies and computer experiments using convolutional neural networks show that computers are as good as or better than humans at one character recognition under all commonly used distortion and clutter scenarios used in todays HIPs. Expand
Building Segmentation Based Human-Friendly Human Interaction Proofs (HIPs)
TLDR
The HIP user studies show that given correct segmentation, computers are much better at HIP character recognition than humans, and it is proposed that segmentation-based reading challenges are the future for building stronger human-friendly HIPs. Expand
Machine learning attacks against the Asirra CAPTCHA
  • P. Golle
  • Computer Science
  • IACR Cryptol. ePrint Arch.
  • 2008
TLDR
A classifier which is 82.7% accurate in telling apart the images of cats and dogs used in Asirra, which is significantly higher than the estimate of 0.2% given in [7] for machine vision attacks. Expand
ScatterType: a reading CAPTCHA resistant to segmentation attack
TLDR
Preliminary results from a human legibility trial with 57 volunteers that yielded 4275 CAPTCHA challenges and responses show that subjective rating of difficulty is strongly (and usefully) correlated with illegibility, and early insights emerging from these data are presented. Expand
...
1
2
3
4
...