Karl Storchmann reports from the front: French wines vs. Jersey wines

by on June 13, 2012 at 4:40 am in Current Affairs, Data Source, Food and Drink | Permalink

At the Princeton tasting, led by George Taber, 9 wine judges from France, Belgium and the U.S. tasted French against New Jersey [TC: that's the New Jersey] wines. The French wines selected were from the same producers as in 1976 including names such as Chateau Mouton-Rothschild and Haut Brion, priced up to $650/bottle. New Jersey wines for the competition were submitted to an informal panel of judges, who then selected the wines for the competition. These judges were not eligible to taste wines at the final competition The results were similarly surprising. Although, the winner in each category was a French wine (Clos de Mouches for the whites and Mouton-Rothschild for the reds) NJ wines are at eye level. Three of the top four whites were from New Jersey. The best NJ red was ranked place 3. An amazing result given that the prices for NJ average at only 5% of the top French wines.

A statistical evaluation of the tasting, conducted by Princeton Professor Richard Quandt, further shows that the rank order of the wines was mostly insignificant. That is, if the wine judges repeated the tasting, the results would most likely be different. From a statistically viewpoint, most wines were undistinguishable. Only the best white and the lowest ranked red were significantly different from the others wines.

There was a third similarity to the Paris tasting. In Paris, after the identity of wines was revelared, Odette Kahn, editor of La revue Du Vin De France, demanded her score card back. Apparently, she was not happy with having rated American wines number one and two.

At the Princeton blind tasting, while both French judges preferred NJ red wines over the counterparts from Bordeaux. After disclosing the wines’ identity the French judges were surprised but did not complain. In contrast, several tasters from the U.S. did not want their wine ratings to be published.

All results and the statistical analysis can be found here:

http://wine-economics.org/WineTastings/Judgment_of_Princeton_no_comments.html

Orley Ashenfelter also refers me to www.liquidasset.com, click on “tastings” and then “the latest” for data.

Adrian Ratnapala June 13, 2012 at 6:33 am

I am surprised that people were surprised by the identity of the wines. Whatever the relative merits of Bordeuax and Big Aussie Red, they tend to taste quite different. I guess either NJ wines taste like Bordeaux, or the ones selected do.

Brian Huntington June 13, 2012 at 7:15 am

I’m not surprised. Penn & Teller did a little “experiment” about self-described connoisseurs being unable to distinguish “The Best” from cheap stuff scraped together by one of their prop guys. I also remember seeing a news clip about “Two-buck chuck” (apparently some kind of cheap wine) winning awards in anonymous taste-tests.

This goes far beyond famous vs. cheap-o wines: Tyler may want to think about whether his foodie habits are just a way of signaling subtlety, openness, and exclusivity. I’m sure that a lot of connoisseurs will make fun of the unfortunates who picked the Jersey wines in this example, but would they be willing to do public taste tests? We should all be updating on this evidence. Do those organic veggies really taste better? Can I really taste the difference between Coke and Pepsi?

Thor June 13, 2012 at 12:56 pm

Brian: Two Buck Chuck might have won awards, but it was in “best budget level wine” categories. It is good for stews, at best.

The wines we’re talking about here are decent. In Canada, owing to gov’t taxes on liquor, most wines under $15 are extremely mediocre, no matter where they originate. Decent starts at a slightly higher price point.

Robert June 14, 2012 at 9:45 am

Coke and Pepsi are easily discernible.

Jim June 14, 2012 at 5:45 pm

I can realy taste the difference between coke and meth.

Steven Kopits June 13, 2012 at 7:23 am

It’s the PCBs that give NJ wine that extra little kick.

Yancey Ward June 13, 2012 at 8:49 am

Thread-winner!!

Thor June 13, 2012 at 12:58 pm

I’m partial to a taste combination of a nose of diesel suffused sand and the lingering aftertaste of “dead gangster” myself.

Marie June 13, 2012 at 8:00 am

Have you tried the 3 cup Pepsi versus Coke test? One cup has Coke, one Pepsi, and the last one has one or the other. Mix them up and the taster has to identify which each cup is. My friends didn’t believe it would be that hard so we all tried it one day – it’s way, way harder than you expect it to be. That third cup really messes with what you think you’re tasting. It’s really difficult to determine which two cups have the same drink in them and which is the odd man out.

I bring it up because our tastes are far less developed than we think.

Andrew' June 13, 2012 at 10:13 am

Will try and report back. But we should keep in mind, these are two things trying to be basically identical. They aren’t even trying to improve. They basically stumbled on something that works.

Plethon June 13, 2012 at 1:04 pm

I’ve done this more than once and still don’t understand how anyone could mess it up. You could throw in RC, too. (I’ve also done regular vs caffeine free when someone couldn’t believe I could tell.)

Enrique June 13, 2012 at 8:24 am

This just goes to show how utterly subjective preferences based on taste or aesthetics are

i found an econ degree on the ground June 13, 2012 at 8:40 am

Or just how provincial the French are.

Turpentine June 13, 2012 at 9:48 am

“Or just how provincial the French are.”

And yet…

“After disclosing the wines’ identity the French judges were surprised but did not complain. In contrast, several tasters from the U.S. did not want their wine ratings to be published.”

Tristan June 13, 2012 at 11:39 am

Taste—yes. Aesthetics—to a degree. Our sense of sight is far more exacting than our sense of smell or taste, and we’re able to make slightly more objective analyses of things we see. Still subjective, but it’s subjectivity based on a far greater accuracy of input.

Benny Lava June 13, 2012 at 8:46 am

Wine is conspicuous consumption. No surprises there. Not sure why Tyler uses terms he invents rather than perfectly good pre-existing ones. I’m sure you will find that whisky is conspicuous consumption as well.

elambend June 13, 2012 at 12:27 pm

I don’t buy wine to show off and I prefer some rather cheaper ones (no, not Thunderbird [or MadDog]). Expensive wine may be conspicuous consumption, but there is an underlying benefit to a nice glass, or a glass.

Jim June 14, 2012 at 5:51 pm

Oh bollocks, as someone who thinks wine is “conspicuous consumption” would probably say. Wine is a grocery item. That’s why God made boxes, and I have had French house guests praise boxed wine.

And whisky is for teething. Hardly conspicuous consumption either.

Yancey Ward June 13, 2012 at 9:10 am

These results could probably be repeated in all areas of professional aesthetic judging.

lords of lies June 13, 2012 at 10:18 am

modern art, like wine snobbery, is a fraud perpetrated for the sole purpose of assuaging the egos of self-styled critics, and offering them a status metric to delineate themselves from the rabble. there was a famous experiment in the news not too long ago which had a splattery “painting” by a chimpanzee examined by various art critics, who were not aware of the painting’s lowbrow-ridged authorship. the result was that they all swooned over each other extolling the artwork’s virtues.

more evidence that it’s been downhill for the west since about 1890. the next few decades will be a mop-up operation.

Eric S. June 13, 2012 at 11:49 am
Urso June 13, 2012 at 1:59 pm

I don’t know about that. The art students could differentiate between the masters and the chimps 60%-70% of the time. So 30%-40% of the time they thought the chimps were better than, or at least as good as, the masters. Does that justify the multiple orders of magnitude difference in price?

lords of lies June 13, 2012 at 2:11 pm

I wrote “art critics”, not people.

ps conflicting studies do not automatically upend a “narrative”. but as an evidenced member in good standing of the closed-minded consortium, i can see why you’d wish that were the case.

sourcreamus June 13, 2012 at 2:58 pm

Telling the difference between a work by a master and one by a child should be an absurdly easy task. It would be like telling the difference between a French Burgundy and a Capri Sun. The real test would be between a master and a talented amateur.

Tom West June 13, 2012 at 9:12 am

What I don’t understand is why judges would agree to a blind testing that endangers their credibility among the cognoscenti and risks damaging the upper-end wine industry.

I would at least expect the experts to understand the game, but it looks like they’ve bought into the myth as much as the consumers, despite the ample evidence that great wines are more or less indistinguishable from good wines. While there’s hundreds of years of tradition and a desperate desire for status keeping the myth alive, if the experts repeatedly lend their credibility to exposing the myth, they’re in danger of helping the whole edifice come crashing down, which would be a massive loss of utility for consumers and producers alike. (To say nothing of the fact that while the myth stands, they damage their personal credibility as wine experts.)

And yes, it would be a pity to lose that myth. Only a small portion of the benefit of upper-end wines is in the consumption, but you need the myth to enjoy all the benefits.

Andrew Edwards June 13, 2012 at 10:44 am

Umm, just throwing this out there, but doesn’t this result suggest that the experts don’t buy the mythos?

They willingly signed up for the taste test, and honestly chose the best wine they tasted, even when it was not “fancy”. Doesn’t that burnish their credentials rather than damage them?

Aren’t these experts now more credible?

Tom West June 13, 2012 at 10:56 am

Except that their credentials are only worth something if you buy into the myth. To exaggerate a little, nobody employs people to write for magazines as soft-drink tasters and they certainly don’t get flown around the globe for competitive tastings.

As I said, consumption of wine at the high end is only a small part of the utility. Strip out all the rest (and that’s what these experts threaten to do) and you lose 90% of what makes wine interesting.

Gary June 13, 2012 at 10:10 am

A big addendum to the results as well. four of the five French reds are from 2004 compared to the Jersey reds which are >/= 2008. That is a HUGE advantage. It is hard to distinguish a lot of things, but older wines are MUCH more mellow and less tanniny than younger wines.

As a Jersey resident who has gone to a lot of those wineries, I say kudos, but now anticipate a steep rise in prices for those wines.

Ray Lopez June 13, 2012 at 11:31 am

There was a similar competition in the 1970s between California and French wines, won by Cali on French soil and a book was written on it a few years ago. As a California wine connoisseur who has never had a NJ wine I can tell this competition was biased in favor of NJ: first they taste white wines, which are easy to fool people (even cheap white wines ‘taste good’), and then it appears they are using cheap red FR wines which may have harsh tones to them. Smells like a publicity stunt.

Urso June 13, 2012 at 1:54 pm

(even cheap white wines ‘taste good’)

isn’t this QED?

scott cunningham June 13, 2012 at 10:17 am

We saw a lot of this in Robin Goldstein’s excellent book The Wine Trials. Goldstein published some of the results from his experiments in the Journal of Wine Economics, but the book is better because it includes the list of cheap wines that did well. My takeaway from Goldstein is that wine has tons of placebo effects, and even so-called experts are not nearly as discerning as we think. I hadn’t seen the effect, though, on top/bottom in the rank order before but I’m not surprised.

Craig June 13, 2012 at 10:19 am

This is why so many wine experts _hate_ blind taste tests. Otherwise sane finance blogger Felix Salmon thinks they’re the height of bad form. You don’t point out the emperor’s nakedness.

Kevin H June 13, 2012 at 10:36 am

It’s easy to tell a good wine from a bad wine but it’s hard to tell good wines apart.

I’ve tasted many wines from NJ and very few would be confused with a French but there are a few good ones that can compete.

P.S. How much of the GMU econ staff was born in NJ? Seems like over representation.

efp June 13, 2012 at 12:23 pm

+1 …and it’s not necessarily “telling them apart” that’s difficult, but ranking them according to preference. It’s basically like a multi-breed dog show… you have to compare each to its own standard–and being a blind tasting, there’s nothing to go on.

Andrew Edwards June 13, 2012 at 10:38 am

I thought this was well understood.

In general, there are inexpensive wines that are just as “appealing” as expensive wines. Who said you were paying for appeal when you buy expensive wine?

I buy expensive wines for two reasons:

1) Consistency. A $40 wine I don’t know is much more likely to be good, and will have a much more predictable flavor profile than a $10 wine I don’t know. The $10 may be good, but it may be terrible, and even if good I have no idea how it will taste. I know what a $40 Napa Cab will taste like, within a fairly tight range of error.

2) Specific flavor profiles. Pizza tastes good, and so does ice cream. But they have very different flavor profiles. Similarly, a well-aged Bordeaux or a Napa Cab taste differently than an $8 bottle from an obscure Italian region, regardless of whether the cheap wine also tastes good. If I am in the mood for the taste of a quality 15-year-old Bordeaux, then I have to pay for it, period.

DK June 13, 2012 at 11:28 am

If I am in the mood for the taste of a quality 15-year-old Bordeaux, then I have to pay for it, period.

What do you want to bet that you won’t be able to tell quality 15-year-old Bordeaux from an unknown California cab?

Andrew Edwards June 13, 2012 at 11:45 am

I’m sure I’d miss here and there, and I may often prefer the cheap cab. But in terms of simply distinguishing one from the other (say in a multiple-repetition Triangle Test) I’d be shocked if I couldn’t beat random chance by a wide margin.

Those wines taste VERY different.

Tell ya what, next time I open an old Bordeaux I’ll set up a test for myself.

DK June 13, 2012 at 6:14 pm

Those wines taste VERY different

Yeah, that’s what the experts that consistently fail to tell one from another always say.

Andrew Edwards June 13, 2012 at 7:51 pm

Just as a check – have you had them? Because they do, in fact, taste very different. And I’ve never seen any evidence to the contrary. These tests that suppose to disprove expert judgement generally focus on experts not preferring the more expensive wines, not on experts being unable to distinguish between them. Watch a few of the videos I linked below – you’ll see blind experts quite often able to nail exactly what they are drinking.

I’m surprised to see you not even consider the evidence here. Is it important to you that “wine snobs” be irrational in their preferences, rather than have legitimate reasons to want to pay more for rarer products?

DK June 13, 2012 at 9:25 pm

I’m surprised to see you not even consider the evidence here.

There is now decent number of scientific-minded studies and then there are some selected videos on a clearly biased web site – and you call the latter evidence? Okay … But let’s go back to my original question: how much do you want to bet that you will be able to reliably tell Bordeaux from, say, Washington?

Andrew Edwards June 13, 2012 at 9:40 pm

Didn’t answer either of my questions. I feel like you’re just arguing now because you want to be right.

I know what I taste. If we were live, we might be able to experiment interestingly. We’re not, so we’re done.

CS June 14, 2012 at 1:38 pm

I’d throw in that there are a number of wine certifications (CWE, various Sommelier tracks, etc.) that require passing a blind tasting exam. Some people can apparently blindly identify wines by grape, region and year and pass these tests. So it is possible. It is probably a rare talent, though, as on the other side you have these hard-to-discount scientific and non-scientific studies (such as the NJ one) that show that most wine drinkers (even experienced ones) cannot tell important differences between wines.

From my own blind tasting experience with friends who drink a lot of wine, I’d say that when a wine with a clear, correct/typical varietal character shows up (like a high-end Cabernet that screams cassis), people can identify the grape. With most wines, though, trying to pin it down much beyond broad categories like old world/new world can be very difficult.

David Cannon June 16, 2012 at 1:13 pm

I got $1k on Mr. Edwards, what say you DK?

DK June 14, 2012 at 5:22 pm

I’d throw in that there are a number of wine certifications (CWE, various Sommelier tracks, etc.) that require passing a blind tasting exam.

I just looked at CWE and nowhere it says that it requires passing a blind tasting exam. It does say that blind tasting (not so blind because it’s multiple choice) is a small part of the overall exam. I bet just about everything that it is possible to get that CWE certificate while not doing better than random on tasting part. (If it were not the case, that particular for-profit operation would have folded long ago).

Andrew Edwards June 13, 2012 at 11:50 am

As an addendum, here is a video series of experts doing exactly this. They aren’t always right, but they beat the hell out of random chance:

http://www.winealign.com/videos

For example:
http://www.winealign.com/videos?v=tBi9PfZve84

Chris June 14, 2012 at 1:49 pm

I’m not sure how well they really do. I just watched a few and they were all way off, like the Perrin Cotes du Rhone one:
http://www.winealign.com/videos?v=HYvBIeMCHxI

David cannon June 16, 2012 at 1:07 pm

I’d bet $1000.

gwern June 13, 2012 at 1:00 pm

> A $40 wine I don’t know is much more likely to be good, and will have a much more predictable flavor profile than a $10 wine I don’t know.

So when you need 2 good bottles of wine, just buy 7 of the $10 ones…

charlie June 13, 2012 at 11:05 am

Isn’t the real result the the “informal panel of judges” is crap.

In other words, bloggers aren’t actually wine experts, they just play one of the internet.

DK June 13, 2012 at 11:22 am

This sort of thing (“experts” not able to tell one from another and no correlation between price and “quality”) was found in practically every blind study ever conducted. These days there are some absolutely great wines sold very cheaply in boxes. They don’t work too well for signaling but work fine for daily dinner. Two of my current favorites:

Pepperwood Grove Pinot Noir (in the nearest large grocery, $16 per for 3L) and Vina Borgia Garnacha 2010 ($18 for 3L)

DK June 13, 2012 at 11:23 am

This sort of thing (“experts” not able to tell one from another and no correlation between price and “quality”) was found in practically every blind study ever conducted. These days there are some absolutely great wines sold very cheaply in boxes. They don’t work too well for signaling but work fine for daily dinner. Two of my current favorites:

Pepperwood Grove Pinot Noir (in the nearest large grocery, $16 for 3L) and Vina Borgia Garnacha 2010 ($18 for 3L)

mthvedt June 13, 2012 at 11:30 am

> “A statistical evaluation of the tasting, conducted by Princeton Professor Richard Quandt, further shows that the rank order of the wines was mostly insignificant.”

I see this a lot when wine skeptics discuss wine, but is it really reasonable to design a wine tasting such that most rankings have statistical significance? You’d need a lot of judges and a lot of tastings.

Stillman Brown June 13, 2012 at 11:33 am

The difference between a judging panel of writers and sommeliers and a judging panel of winemakers is considerable, and a judge who can’t tell the difference between a median quality $1/liter bulk wine and a median quality $50 bottle should perhaps be doing something else.

dearieme June 13, 2012 at 1:11 pm

35 years ago my wife and I used occasionally to organise wine tastings for various groups. We found that if we selected, say, nine good wines at about 6-9 GBP per bottle (modern price) and one at twice that, then always the majority of the tasters could pick out the expensive one. And, by a substantial majority, they preferred it. I can’t swear that doubling all the prices would show the same effect. Nor can I swear that the test would work nowadays with a population more used to slurping American sugar-water. But I can promise you that when people tasted a good Cote Rotie – for example – for the first time, they raised their eyebrows and then raised their glasses for more.

Roy June 13, 2012 at 3:19 pm

Exactly, I drink in the $10-14 range, because that is the most I can really afford, but whenever I by anything better I can tell, even if I mix the bottles up. I look forward tothe day I can move up to the $20 range. I will admit though that I detest sweet wines and have little love of Cali and Aussie style fruit bombs, so I might be drinking from a different pool than others.

Doug Charles June 14, 2012 at 5:48 pm

As someone in the trade, who tastes literally hundreds of wines per month, I will be the first to admit that all of us get stumped on varietals on a regular basis. The real variable is not what is ‘better’, it is what is ‘drinking better’ at that moment. Sometimes vanilla tastes better than strawberry and sometimes Cab tastes better than Pinot. To say that wine experts can be duped when judging is like saying that movie critics sometimes get it wrong. duh! Why does it matter if a wine geek like me can’t tell a South American Tannat from a Loire Valley Cab Franc on a given Thursday? if it tastes good at that moment, does it really matter? I frankly don’t understand why there seems to be this innate need to trip up judges of subjective material. Doesn’t the fact that it is entirely subjective tell the whole story? My favorite pie may be blackberry, but you may prefer cherry. Even if I were a pie ‘expert’, does that make one of us wrong?

DK June 14, 2012 at 8:30 pm

Even if I were a pie ‘expert’, does that make one of us wrong?

If you are a pie expert and can’t tell blackberry from cherry despite the fact that your whole career is built around the claim that you can, it makes you a fraud. Fraud is very common in human endeavors.

Doug Charles June 15, 2012 at 1:45 pm

my question is not whether we can tell the difference, but the idea of what makes one product ‘better’ than another. If the judging was to just tell them apart, without regard for which was preferred, I would agree with you. But the judgings are always about which the judges prefer, not about telling them apart. Those of us who do wine for a career look at the whole issue in a much different way than the public. Our job is not to stand on a pedestal to tell the world that we can tell one wine from another, but to guide prospective buyers into what we think they will LIKE better. A way different concept. If all you want is some machine to tell if a grape is grown in sandy loam outside of Jersey City or in clay and gravel in Graves, I am sure you can program a computer to do that. If you want an opinion on what we like or dislike, then ask a wine pro.

Comments on this entry are closed.

Previous post:

Next post: