Seeing is Hearing: The McGurk Effect

Very cool auditory illusion created by visual dominance.

Hat tip to Carpe Diem.


This conclusively demonstrates the Great Disadvantage of face to face social relations.

This was partly tongue-in-cheek by the way (but you can't see that).

+1 for both comments. lol.

Aha! Holonic (expectation-influenced) interpretation at work. For a 15-minute talk about an AI architecture that works this way (by yours truly), see

Last year, I've visited quite some anglo-saxon economic blogs. To my surprise, many fo them still cling to a rather outdated and refuted concept of rational, optimizing, atomistic man (a concept ridiculed by, for instance, Hayek). This is a very welcome antidote. We're not independent atomicons. NIce to see this on a blog like this, we're making progress.

Now I have to wonder if I'm the only person in the world who is (mostly) immune to the effect.... I _do_ see the "fa", and I _do_ start to hear it. But something kicks in (very rapidly) & overwrites my perception to "ba". I have to pay careful attention, or I can miss the "fa" entirely. My initial response was "why is his mouth going 'fa' while he's saying 'ba'?"

For me, I ended up thinking, "That sounded like ba, but he said fa. His lip must be flapping unnecessarily, and he's actually trying to say fa."

When the two side by side are together is when it gets the most interesting. If I don't focus on either person, my brain says it's the right-hand ba speaker that's talking.

You're not alone. Throughout the last 2 minutes I wondered: "What's the illusion?" I heard "fa" immediately. FWIW, I'm a radio junkie.

I couldn't tell the difference between the two 'sounds' either. It _almost_ sounded like /pa/ or /da/ or /fa/ at various points, but I still 'concluded' I was hearing /ba/. Maybe this is related to my having a hard time 'hearing' song lyrics; to me they're more of another (musical) instrument, though I still perceive them as voices.

Very cool.

So now I know why all those live concerts sounded so great in person but so crappy on tape. It wasn't just the, uh, aromatic enhancers. Musicians have always known that you hear a concert with your eyes.

Outstanding comment.

Terrible comment.

Spiteful comment.

Commencing comment...

I have read and had this demonstrated before, but it was interesting to, first, close your eyes and listen to the entire segment. Without question, it was ba throughout. Then it was interesting to watch the fa motion for a few seconds, then close the eyes. At first, I kept hearing fa, but after a couple times, it reverted back to ba. Very strange how the brain works.

Doesn't work with me. I've heard "pa" throughout.

What's your native language?

Exactly the right question, seeing this I remembered taking a Chinese class with two Japanese girls many years ago, their complete inability to distinguish between ba and pa, combined with their utter commitment to the program actually contributed to the wrecking of an academic department at a major midwestern university.

Not Wisconsin, by any chance?

Coolest thing I've seen on MR this month!

The problem with this is that the subject is mispronouncing "Va."

We are interpreting "Ba" when we view an abbreviated "Va." But if the subject were stressing the "V" of "Va," we could discern the problem, visually.

The problem is the structure of the experiment; not our interpretation.

As an old school perceptual realist, I have to say that's an amazing illusion. I wonder if JJ Gibson knew about this one...

I found the url buried somewhere. cancel previous comment

When I closed my eyes, I was able to make myself hear either one by recalling the image.

Exactly Brian G, this is the best way to beleivei it;s happening.

This is very culturally dependent. I am not a linguist, but I know that our pronunciations of "b" and "v" are tied to our northwestern European antecedents.

In Russian (and maybe other Slavic languages), the b and v sounds are almost identical. The Russian word for "you" sounds like "bvwee."

And in the Mexican iteration of Spanish, "b" and "p" are very similar. (Um, my colloquial example is rude, but very commonplace. Let's just say it is sphincter-oriented.)

I know speech therapists are very educated in all this, as are linguists. Something about "aspiration," about whether air is moving in or out of our mouths or is static as a sound is uttered. Certainly, the positioning of lips is critical, but that is seldom the only factor in cognition, just an enhancement.

I suspect that, as in so many social interactive events, it's all about context. We (or "our brains" as the video explains) rarely take in clues from only one sense as we process information. Besides auditory input, we gather visual clues, we evaluate environmental, experiential and social settings, olfactory cues, distractions, social settings, all so quickly and without conscious thought (unless trained to employ conscious evaluative techniques.).

So "our brain" decides whether the consonant is "b" or "f" or "v" or "p" based on not only the sound we hear and the lip motion we see, but also the context of the words expressed, the social or physical environment (amusement park, hospital or courtroom?), possibly a perceived accent, the "noise" factor (Whether actual environmental noise or the cultural "noise" of an unfamiliar country, etc.).

Billions of neurons are furiously calculating, trying to make sense of disparate inputs, to answer our (our brains') constant questions: What's happening now? Am I threatened? What do I need to do/know to survive/thrive/conquer? Can I relax yet? Should I fight/flee/surrender?

In the context of all these factors, the McGurk effect is interesting to Academia, but the real question is how often it actually affects or confuses our decision-making process regarding any individual's survival.

In any instance of confusion of a spoken statement, how frequently does the confusion of "b" and "f" actually result in misunderstanding vs. how frequently do additional clues and inputs result in a correct interpretation of the statement? That's the key.

If in large part we understand correctly the message being expressed, the communication was successful despite some contrary visual clues.

(For more context, I am amdues by teh expamlse of mseasges whit srcbamlde ltestre oru bniasr cna dceode!)

Rex - while Mexicans may be merging 'b' and 'p', pretty much all Spanish-speakers have merged 'b' and 'v'. So much so that the Spanish Academy decided that the name of the letter 'v' would henceforth be "uve", which was not universal among Spanish-speakers, to eliminate confusion between "be" and "ve" (the more widespread name of the letter 'v'). Also notice that Spanish-language maps often spell the name of Cuba's capital city as "Habana", which is not phonetically distinct from "Havana".

Comments for this post are closed