• DOI:10.1145/175208.175217
• Corpus ID: 215823854

Lessons from a restricted Turing test

@article{Shieber1994LessonsFA,
  title={Lessons from a restricted Turing test},
  author={Stuart M. Shieber},
  journal={Commun. ACM},
  year={1994},
  volume={37},
  pages={70-78}
}

• S. Shieber
• Published 3 April 1994
• Computer Science
• Commun. ACM

We report on the recent Loebner prize competition inspired by Turing's test of intelligent behavior. The presentation covers the structure of the competition and the outcome of its first instantiation in an actual event, and an analysis of the purpose, design, and appropriateness of such a competition. We argue that the competition has no clear purpose, that its design prevents any useful outcome, and that such a competition is inappropriate given the current level of technology. We then speculate as to suitable alternatives to the Loebner prize. This report appeared in Communications of the Association for Computing Machinery, volume 37, number 6, pages 70-78, 1994. Also available as cmp-lg/9404002 and from the Center for Research in Computing Technology, Harvard University, as Technical Report TR-19-92. The Turing Test and the Loebner Prize The English logician and mathematician Alan Turing, in an attempt to develop a working definition of intelligence free of the difficulties and philosophical pitfalls of defining exactly what constitutes the mental process of intelligent reasoning, devised a test, instead, of intelligent behavior. The idea, codified in his celebrated 1950 paper ``Computing Machinery and Intelligence'' [28], was specified as an ``imitation game'' in which a judge attempts to distinguish which of two agents is a human and which a computer imitating human responses by engaging each in a wide-ranging conversation of any topic and tenor. Turing's reasoning was that, presuming that intelligence was only practically determinable behaviorally, then any agent that was indistinguishable in behavior from an intelligent agent was, for all intents and purposes, intelligent. It is presumably uncontroversial that humans are intelligent as evidenced by their conversational behavior. Thus, any agent that can be mistaken by virtue of its conversational behavior with a human must be intelligent. As Turing himself noted, this syllogism argues that the criterion provides a sufficient, but not necessary, condition for intelligent behavior. The game has since become known as the ``Turing test'', a term that has eclipsed even his eponymous machine in Turing's terminological legacy. Turing predicted that by the year 2000, computers would be able to pass the Turing test at a reasonably sophisticated level, in particular, that the average interrogator would not be able to identify the computer correctly more than 70 per cent of the time after a five minute conversation. On November 8, 1991, an eclectic group including academics, business people, press, and passers-by filled two floors of Boston's Computer Museum for a tournament billed as the first actual administration of the Turing test. The tournament was the first attempt on the recently constituted Loebner Prize established by New York theater equipment manufacturer Dr. Hugh Loebner and organized by Dr. Robert Epstein, President Emeritus of the Cambridge Center for Behavioral Studies, a research center specializing in behaviorist psychology. The Loebner Prize is administered by an 1 of 14 03/03/99 17:24 Lessons from a Restricted Turing Test http://www.eecs.harvard.edu/shieber/papers/loebner-rev-html/loebner-rev-html.html