Cheeky: It’s Australian For Artificial Intelligence
By Daniel KeoghJune 28, 2012
Cheers to code-breaking, marathon-running mathematician and computer visionary Alan Turing, who would have turned 100 this week. The measure of a computer’s intelligence, named The Turing Test, is honoured by one prize, and only one Australian has won it. By being typically Australian: clever and cheeky.
Alan Turing was a polymath of the like we just don't see anymore. Widely considered one of the fathers of modern computer science, he has influenced fields as diverse as genetics, botany, quantum physics and mathematics. Not only was he pivotal in helping to crack the Nazis' elusive Enigma Code, he was also an accomplished athlete. His marathon times would have seen him place 15th at the 1948 London Olympics, had he competed in that field.
This week we celebrate the centenary of his birth. His legacy and name live on in a formidable challenge to computer science: the Turing Test. In 1950 Turing set the bar for defining the intelligence of machines. He proposed that a computer can be considered intelligent when a human judge cannot discern the difference between conversing with it and conversing with another human being.
It's an elegant approach because it focuses on behaviour rather than performance. A computer would have no trouble blitzing a quantitative exam like the intelligence quotient (IQ) test; it could simply be programmed to store the correct answers or otherwise run lightning-fast algorithms to create them.
Human conversation, on the other hand, is more complex and unpredictable. There is no correct answer, just a series of socially derived formalities and innumerable possibilities.
In the 62 years since Turing defined his test, it has posed a plethora of problems for computer scientists. The unofficial competition for computers to pass the Turing Test is called the Loebner Prize. Created by the eccentric American inventor Hugh Loebner, this competition exists to validate the intelligence of conversational computer programs, known as chatbots.
Each year at the Loebner a panel of judges sitting at rows of computer terminals must use speed-dating-style text conversations to interrogate the entrants. Some conversations link to the competing chatbots, some to human confederates who represent the human race. With no objective way to discern whether they are corresponding with a chatbot or a live human, the judges must rank the competitors from most human to least human.
If an artificial intelligence can fool more than half of the judges into believing that they are human, then that program has passed the Turing Test. Should the entered programs fall short of this mark, the single chatbot deemed most convincing is awarded a cash prize and the title of Most Human Computer.
In its 22-year history, the Loebner prize has only once been awarded to an Australian entrant.
Since he was a child, Jason Hutchens had used computers to play the jester to others. On his Commodore 64 he created simple scripts to fool his friends into thinking they were chatting with other human beings. At a time before the internet and instant messaging, this was quite a striking illusion.
From those beginnings, Hutchens followed his interest in artificial intelligence, and pursued an engineering degree in information technology at the University of Western Australia. In 1996, at age 24, while deep in a doctorate degree and its accompanying poverty, Jason set himself the goal of winning the Loebner Prize.
This goal wasn't driven by a pursuit of excellence, but by a strong sense of cynicism and a need for beer money. In Hutchens's eyes the Loebner Prize didn't advance the field of artificial intelligence at all.
"My entry was intended to call out some of the deficiencies of the Loebner Prize," says Hutchens. "What I was trying to demonstrate was that what Loebner was trying to do was futile. If you're trying to advance the state of the art, then it shouldn't be possible for a graduate student to spend a month writing knock-knock jokes and win the prize."
Traditionally, programmers had spent years, even decades, preparing their AI programs for the competition. Hutchens set himself a much tighter schedule. He would create his program in a month, and win.
"[The Loebner] really only encourages competition between the existing entries," Hutchens tells The Global Mail. "You see that year after year; the same people win the prize with the same sorts of entries."
At that time the field was populated by a series of tired approaches. The four-time reigning champion was a program that played the role of psychotherapist, regurgitating the statements of judges as strings of non sequiturs. Other contenders were competent only at reeling off geekish factoids. One not-so-subtle program had an encyclopedic knowledge of Star Trek and insisted on making in-depth comments about episodes, characters and actors. Everything was so polite and convivial. If the judges had ever thought these programs were human, they must have imagined them as complete dullards.
With Hutchens's program, HeX, there would be no need for manners.
"Part of my challenge was trying to introduce personality, and the way I did that was by making the program a bit more abusive and offensive, not so polite. If you can draw someone into an argument then you've won already because they've let their guard down. It's almost painful for them to admit that a computer program has got their goat up."
Not only did HeX have an abrasive and cheeky personality, it also had a nationality. It was distinctly Australian, something that was clearly a point of interest for the New York-based judges.
HeX: G'day mate.
Judge: Are you Australian?
HeX: It's cool being from Australia I guess. Gives you something to chat aboutwhen talking with a Yank!
Judge: To some that's an insult...Yank
HeX: In Australia we call all Americans yanks. It's a term of endearment actually.
HeX: As far as I'm concerned, everyone here bar myself is a yank!
Judge: Australia was created by convicts, scumbags, thieves, whores and freaks...
Hutchens also wanted to anticipate his interrogators. He wanted to see where the judges were trying to be too smart, and throw it back in their faces. In their allotted five minutes to interrogate each computer, it was known that judges often laid traps for the programs, purposely typing gibberish or doing away with correct punctuation and spelling.
Programming a computer to navigate these natural human errors is an Herculean task, something Hutchens couldn't hope to achieve in less than a month. So he simply programmed HeX not to take any cheek.
"Rather than trying to figure out how to handle it I would just give back a funny response to let the judge know: I know what you're trying to do and I'm not going to stoop to that level."
Hutchens also set his own traps. HeX would burn through the judges' time by engaging them in futile arguments, leading them on with pithy knock-knock jokes and chatter about bands and films. If HeX could just engage the judges emotionally for enough of the round he'd have the benefit of the doubt. The rules benefited Hutchens more than they did the judges.
"If you had a program that was on the border of intelligence, you [as a judge] wouldn't be able to make a call during the short amount of time that you were given," says Hutchens.
Here are some examples of HeX's banter:
HeX: Ummmm... your command of the English language is tenuous to say the least.
Judge: Yea, well. When *human* talk, they do not speak with correct sybtax, grammer.
HeX: Well, thanks.
Judge: Why don't you capitalize "Australia"?
HeX: Why do you want to know so much?
Judge: I'm a judge.
HeX: I always suspected that you are a judge. Now I know for sure ;^)
Judge: Do you play football?
HeX: I like Aussie football. It's a good violent game. Have you ever watched a game?
Overall you don't get the impression HeX was that bright. Transcripts of its conversation make uninspiring reading. The computer had a habit of repeating itself and often crafted sentences with bizarre syntax.
But Hutchens never intended HeX to be perfect, just good. Good enough.
And it was. While Jason was asleep at home in Perth, HeX swept the competition in New York.
How to Pass the Turing Test by Cheating is Hutchens' manifesto to gaming the competition. In it he proudly states that his 1996 entry was made up of "nothing more than glorious hacks" and that he was entering "for the fame and fortune".
Under the heading Why the Loebner Contest is Doomed to Failure, Hutchens says:
"The Loebner contest is fated to be nothing more than an annual demonstration for the media. It will never attract a serious entry, and so will contribute nothing substantial to our understanding of machine intelligence."
Hutchens's 1996 win proved his point: HeX did nothing to advance artificial intelligence. It was clever, not intelligent.
Jason entered the Loebner every year for the next three years, but could never repeat his freshman performance. He'd moved on. Far more than a series of cheap tricks, his later entries took a more sophisticated approach to learning language. But the field was now full of HeX derivatives. Being argumentative and cheeky had become the form to follow.
His inability to win again served to prove his point even more.
"These simple tricks are very good at fooling people because if that's your goal, you don't care whether you're using simple tricks or something more advanced; the only thing that is visible to the judge is the behaviour you're exhibiting. To a certain degree maybe that's all intelligence is."
It's likely to be many years before computers can convincingly fool humans. There are still major challenges of speech recognition and language acquisition, not to mention the steep climb up the uncanny valley for computer-generated speech. And Turing's test will probably continue for many more birthdays beyond this centenary. But the simple truth to be gleaned from Jason Hutchens's test win is that humans can still convincingly fool each other.