Return to MODULE PAGE
The Turing Test (w/o Suggestions)
David Leech Anderson: Author
Kari Cox: Animations, Storyboards
The Turing Test for Machine Intelligence
In 1950, Alan Turing introduced what has come to be called the "Turing Test" for machine intelligence. The test involves two subjects, a human and a machine (the computer to be "tested") who engage in conversations with some number of interrogators. Each interrogator (human beings) will be placed in a room with a computer terminal. Using the terminal to communicate, each interrogator will engage in two conversations with each of the two subjects -- the computer (to be tested) and the human subject. The interrogators do not know which of the two subjects is the machine and which is the human. It is their job to ask questions or to say anything in conversation that might trip-up the computer program and identify it as the machine. After the interrogator has a conversation with both subjects, the interrogator must guess which is the person and which is the machine. Turing never specifically states "official criteria" for what counts as passing the test. However, he describes a certain level of accomplishment that he believes would be reasonable to expect within 50 years:
"It will simplify matters for the reader if I explain first my own beliefs in the matter. Consider first the more accurate form of the question. I believe that in about fifty years time it will be possible to programme computers with a storage capacity of about 109 to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. The original question, 'Can machines think?' I believe to be too meaningless to deserve discussion. Nevertheless I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. ." (Turing, A.M. (1950). "Computing machinery and intelligence." Mind, 59, 433-460 -- Qt from Section 6)
In this quote, Turing did not say that he is giving official criteria for the test. He did say, however, that he believed that by the year 2000, there would be computers capable of reaching the 30%+ threshold of false guesses after 5 minutes and that (as a matter of psychological fact) people would find it natural to call such impressively behaving machines, "intelligent."
Below is an animation that shows how an actual Turing Test might be set up. This version of the animation is "without suggestions" which means that no sample questions will be offered that might be used in a Turing Test. That is entirely your job. If you wanted to trip up a computer so that you could distinguish it from a human speaker, what questions would you ask? Make a list. And explain why you think it would be hard to program a computer to give an intelligent-sounding answer to each question.
There is great controversy about whether this is or is not a good test for machine intelligence. Not all those who think it is a good test, agree on the reasons why. There are some who defend the Turing Test as a good test for intelligence because (1) they believe that the test demands a special kind of behavior, and (2) they believe that intelligence consists in being capable of behaving in just that way. There are others who think it is a reliable test for intelligence but they do not believe that intelligence can be said to consist in a certain kind of behavior. Rather, they believe that intelligence is essentially something more than just behavior, yet they believe that intelligent behavior is a reliable indicator of the presence of that other essential feature because (given the laws that govern this universe) you will never get that kind of behavior without that presence of the essential property.
FOR FURTHER READING: John Searle's (in)famous "Chinese Room Argument" is a direct assault on the Turing Test.
