Can you pass this Turing test?

by on August 6, 2012 at 3:00 am in Education, Games | Permalink

What did they think about the weather that morning?

Three different responses came from a male human, a female human and a machine. Which is which? Keep in mind that the event was held in October 2008 and they all knew it was autumn/fall in England. The responses were:

A.”I do tend to like a nice foggy morning, as it adds a certain mystery.”
B. “Not the best, expecting pirates to come out of the fog.”
C. “The weather is not nice at the moment, unless you like fog.”
So which is which?

That is from a paper by Kevin Warwick, “Not Another Look at the Turing Test.”  I will offer the answer when I get back home.  For the pointer I thank Michelle Dawson.

Julian August 6, 2012 at 3:07 am

I would say a) came from a woman b) from a man and c) from a machine.

Ronald Brak August 6, 2012 at 3:50 am

I’m Australian and my intuition is that b) is more likely to come from a British woman than a British man. (But I intuit nothing on whether the reply is more likely to come from a male machine or a female machine.)

B August 6, 2012 at 1:17 pm
Maximum Liberty August 6, 2012 at 9:50 am

That was my reaction, because only the last item is a complete sentence using straightforward grammar. But, if I knew that any of these statements was a quotation from an obscure book, then I’d say that is the computer. If it was from a fairly well-known book, then I’d say that is the woman. (Sorry, British gents.)

Max

Rahul August 6, 2012 at 11:32 am

I thought so too but guess I’m wrong. I followed the paper to try out the annual winner software at the Turing Contest. Those seem surprisingly bad at mimicking humans. Not sure how they do so well in these contests.

Here’s two AI Machines that won the Turing Contest last two years if someone wants to try their skills as a human interrogator:

http://bit.ly/Jabberwacky

http://bit.ly/Ultra_HAL

merci August 6, 2012 at 3:53 am

I think that b) comes from a machine, c) from a man and a) from a woman.

prior_approval August 6, 2012 at 3:54 am

Well, I just googled the phrase “I do tend to like a nice foggy morning, as it adds a certain mystery.”, avoided the PPT first hit, and found the answer to the puzzle in google books.

Which pretty much means that the question (and answer) to AI is very unlikely to be something involving the Turing test, and likely something more along the lines of a globe spanning data network, consuming megawatts of power, meshing a measurable percentage of the globe’s entire computing/storage capacity. And which will also involve meshing various elements into a whole – much like our intelligence is intricately tied up into the correct functioning of cells located in places like our pancreas or thyroid – neither organ being customarily thought to involve thought or consciousness, but essential to their existence in ourselves.

Johnz August 6, 2012 at 2:27 pm

So since I just googled it to find the answer….did I pass or fail the test? :)

prior_approval August 7, 2012 at 7:42 am

Neither – you are just part of a larger ‘organism’ which is currently in its very early stages of incorporating information whose continually expanding scope is beyond that possible for individuals – regardless of whether the ‘individual’ is defined as a single person or a single machine.

The Turing test is essentially based on an individual human evaluating an individual machine – a very quaint notion from almost two human generations ago. Anyone want to speculate how many machine generations it has been since the first network was created between them?

Doc Merlin August 6, 2012 at 4:06 am

Grrrr, humans don’t “pass” touring tests.
A touring test isn’t even a test. The idea behind a “touring test” is that if you can pretend to be intelligent, then you are. It is just a possible answer to the age old question of what actually is intelligence.

JWatts August 6, 2012 at 12:08 pm

“Grrrr, humans don’t “pass” touring tests. A touring test isn’t even a test. The idea behind a “touring test” is that if you can pretend to be intelligent, then you are.”

And if you are a human and you can’t successfully pretend, does that mean you “fail” the test. ;)

Alex Godofsky August 6, 2012 at 4:42 am

The Turing Test Does Not Work That Way. In particular, if you’ve got a computer that can just data mine a repository of complete human sentences for what looks like an appropriate response to a question, your Turing test must allow the judge to choose the question. If you take an isolated question/response and ask the reader to judge it then you aren’t going to get useful information (unless it’s a question the computer got absolutely wrong).

Colin August 6, 2012 at 10:10 am

Correct, the complete Turing test requires a back-and-forth between the tester and the subjects (individually) – a conversation. Isolated questions, devoid of context, are not sufficient. If, at the conclusion of the conversation, the human tester cannot tell the difference between the human subject and the computer subject, then the computer “passes”. The human tester isn’t the one to pass.

Tom West August 6, 2012 at 11:20 am

I always that that a human failing the Turing test meant that they were unable to convince people that they weren’t a computer.

I’ve had days like that.

dearieme August 6, 2012 at 6:13 am

Here’s a suggestion for the development of a Turing test. Humans (at least alert humans) can often tell that a joke is intended even when they don’t “see” the joke. Can AI machines?

Doc Merlin August 6, 2012 at 8:16 am

Yes.

Doc Merlin August 6, 2012 at 8:17 am

You are missing the point of the Turing test. The Turing test IS NOT A TEST its a definition of intelligence.

dearieme August 6, 2012 at 1:21 pm

You’ve just failed, Doc. I accuse you of being a machine.

Scott August 6, 2012 at 6:36 am

Woman, Man, Machine. B is the most interesting. It’s an attempt at humor, which is far more likely to be a man than a women. But, at face value, it’s expressing fear, which most men wouldn’t do in this situation.

Andrew' August 6, 2012 at 7:26 am

How do we know the respondents were intelligent people?

There is a two-way Turing street here.

Could a robot tell who was a robot?

J. Goard August 6, 2012 at 8:01 am

I feel like an Englishman would have a much better sense than I do, but (b) sure sounds like a man’s sense of humor, and (c) a woman’s.

Slocum August 6, 2012 at 8:21 am

You could guess, and you’d even have a 1 in 3 chance of being right — but all three are plausible human answers AND machine answers to an common question that could be answered with a bit of searching. A) and especially B) are the kinds of canned answers I’d add to my Turing Test candidate system to fool people, and Tyler wouldn’t be highlighting this if the answer wasn’t somewhat counter-intuitive, so by that logic…either A or B is the computer answer — probably B.

I have given phone interviews, BTW, to candidates who, I quickly assumed, were googling answers to technical questions. It wasn’t difficult, though, to come up with non-standard variations that made their lack of any flexibility or depth of knowledge pretty obvious. So they failed their ‘turing tests’.

rick August 6, 2012 at 8:35 am

The machine is C.; why would a machine associate fog with mystery, and how would a machine be imaginative enough to envision pirates coming out of the fog?

chakira August 6, 2012 at 8:55 am

LOL @Tyler trying to prove he knows what a turing test is after his use of the term in the Manilla post.

Ryan Cousineau August 6, 2012 at 1:17 pm

He’s previously used “Turing Test” as a shorthand for being able to understand a debating opponent’s arguments well enough to explain them fairly, or something similar. The Manila comment was an in-joke.

As Turing originally proposed the test, the interrogator’s assigned goal was to tell if the subject was a man or a woman, because the real challenge is not for the computer to be more human than humans, but rather to not give itself away if granted a presumption of humanness. To make another Cowenian reference, very Straussian.

(by this standard, even the super-crude Eliza program often fooled naive interrogators into thinking it was human for quite a long time. There’s an xkcd comic about AI as an emergent result of comment spam that is relevant here.)

IVV August 6, 2012 at 8:57 am

A.) Machine. Talking about foggy days in general, not about today.
B.) Man. Pirates are occurring to him. His world sees more competition.
C.) Woman. Process of elimination more than anything.

Dan August 6, 2012 at 9:18 am

I concur.

Kyle Johansen August 6, 2012 at 10:10 am

This was the same reasoning and answers I thought too. Answer A doesn’t answer the question.

DW August 6, 2012 at 11:10 am

I knew with certainty that (a) was a machine, but I didn’t know why until I read your comment.

FredR August 6, 2012 at 9:10 am

Woman, machine, man.

Mike D August 6, 2012 at 10:12 am

A) Man – expresses optimism, links to portrayal of fog in pop culture
B) Machine – A bit absurd, as if generated by a random connection in a database of fog mentions
C) Woman – A bit sardonic and matter-of-fact

I cannot honestly say why I’d put A&C in their respective places, just my personal observation of what men and women express generally. Probably says more about me than the respondents.

Only strong conviction I have, however, is that the woman did not say B.

Hari August 6, 2012 at 10:15 am

A. Machine. ( Sounds more like a written answer than a spoken answer)

B. Man ( Sense of Humour. I dont know, who has more sense of humour. British men or women)

C. Woman ( Somehow, i think, ‘Unless’ is used by women more )

Walt G August 6, 2012 at 10:37 am

A) Machine
B) Woman
C) Man

Yancey Ward August 6, 2012 at 11:23 am

C seems grammatically correct, and more likely to have come from a machine. A sounds a bit pompous, and I assign it to a man.

Ed August 6, 2012 at 11:24 am

Actually I agree with Mike, and this is what I came up with instantly on seeing the problem. But like him, I don’t have strong convictions on the answer.

Kenneth W. Regan August 6, 2012 at 11:47 am

The Pirates! — hence agree with Mike D and Ed and some others above.

Kenneth W. Regan August 6, 2012 at 11:49 am

The Pirates! (got a “slow down!” message owing to ‘ for ” typo in link)(and again because maybe it suspects the same typo in my previous parens)

Sandman August 6, 2012 at 2:28 pm

B involves humorous whimsy. It involves the concept not just that pirates often appear out of the fog in movies etc. but also that it is a funny thought for that to happen. I would expect a computer to say something more like “not the best, expecting motor bikes to come out of the fog”
C involves irony / holding both the concept that “I don’t like fog” but also that some strange people do like it.

A is just a statement that there is fog and it is mysterious and the speaker likes it. It seems like a basic response “Fog is mysterious. Mysterious things are appropriate to be liked, so I like fog”

B and C are both more human responses. B the most human.

Enrique August 6, 2012 at 3:31 pm

I would like to see the Turing Test applied to law

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1978017

Doc Merlin August 6, 2012 at 7:12 pm

Wait a system that requires people to understand (or at-least pretend to) the law before applying it? It will never happen. There is just too much law.

Willitts August 6, 2012 at 7:34 pm

Judges start from their desired end state and argue backwards toward precedent or a reason to break from precedent. Often, stare decisis is actually stare lazialis. The quickest way to clear your docket it to dismiss everything based on some flimsy precedent and deal with it on remand if the court of appeals is less lazy than the judge.

Dje August 7, 2012 at 12:07 pm

C) Machine- Only logic is required for this answer so I think it’s the machine.
B) Woman- Man will not admit being apprehensive about pirates, so this must be the woman’s answer.
A) Man- The remaining answer must be the man’s.

Gi August 7, 2012 at 12:34 pm

B sounds like a woman to me, c like a bloke and a like a machine. But it’s a bit culture dependent, as the first post says, b is an English woman.

Nelson August 7, 2012 at 10:47 pm

A. Woman – A bit romantic in the thinking.
B. Man – Seeing adventure and warfare in common weather.
C. Machine – Formal in both logic and structure.

Ross Parker August 8, 2012 at 4:55 am

This Englishman would go with:

A) Machine – it is not a direct answer to how the weather is, it’s just a comment about weather preferences.
B) Human, maybe woman (as quite imaginative)
C) Man, maybe man (as quite sarcastic)

I am more confident in the human/machine divide than the male/female one. I guess the latter isn’t the point of the ‘test’ anyway. And yes, I know a Turing test is not a test for humans, it’s a standard of intelligence. And a further yes, it’s best applied to conversations where the interlocutor picks the question. But this is still fun.

Strontium August 15, 2012 at 8:21 am

Did I miss it? the answer is….?

Comments on this entry are closed.

Previous post:

Next post: