Tasting price

A report on last week’s tasting, structured to approximate an experiment. Do please try this at home.


Background and Relevance

We had the cheek to title our tasting last Thursday “Does Price Matter?”, which of course is a question whose theoretical underpinnings have been debated by wine writers, and whose real-world effects have been charted by economists and investigated by psychologists, for generations. Our aims for this particular tasting were to entertain ourselves with a hands-on demonstration; and to add yet another drop to the accumulating ocean of data on the topic. It’s an exercise we hope to repeat occasionally in the future.

If you, like me, perk up at the mention of “wine” in any news headline, you’re aware by now of the plethora of research on wine price and its effect on consumers’ enjoyment. Not only are wine and money both sexy topics, but this particular question prods at some of our more vulnerable parts—connoisseurship, snobbery, geekdom—our need to believe that there is *something* behind our hobby. I find it exciting that the same research report may be spun in the media either with triumph as an exposé meant to debunk the “myth” that expensive wines are worthwhile, or more placidly as a demonstration of the powerful effect of suggestion on human sensual perception and emotion. Or a piece written in one manner may be interpreted by readers in the other—depending on the baggage with which the reader approaches the topic.

Take, for instance, the incendiary Observer article from last summer, whose title leaves nothing to the imagination or to the reader’s own interpretation: Wine Tasting: it’s junk science

In contrast, a focus piece on the Science Friday podcast last week on “Wine Psychology” (talk about good timing!), which explores “how expectations, environment, and social cues can fool us into believing that our wine tastes better or worse than it is”. Take a peek at the comments and the producer’s responses—an indication, I think, that one doesn’t need to be a skeptic of wine tasting to enjoy exploring the mechanisms that mess with a wine-taster’s objectivity.

And some more background reading, should you be interested:

Why Wine Costs What It Does - An article from the New York Times with a fascinating overview of the costs that go into a single bottle of wine, from the grapes to the cork to the marketing. Unfortunately, the numbers may be a little off, as it’s from 2003.

The Price of Wine - A very thorough post on the Priceonomics blog. Some excellent information both on the cost of making wine and our perception of the price.

Wine Market Council Research - Some cool graphics on the economic shape of US wine consumption.

Do More Expensive Wines Taste Better? Evidence From a Large Sample of Blind Tastings - A working paper by the American Association of Wine Economists. Another one of these studies, using a very large dataset; this is exceptionally clearly written and a good read. Importantly, they separate “expert” wine drinkers from those who have little experience; also, they model ratings on the log of the price (because a difference of a few dollars is more important at lower prices than at higher prices). They find no significant correlation between price and rating among the non-experts, but a significant effect in the experts.

Judging wine quality: Do we need experts, consumers or trained panelists? - One of the research papers we referred to during our discussion of “quality” on Thursday—a very in-depth study that is less about price and more about consumer perception and consistency of judging quality.

Details on the Thursday evening tasting:



The format was simple: 5 bottles of red wine, all the same varietal from approximately the same region and similar vintages, but with per-bottle prices that spanned a range of about an order of magnitude. Nobody was given any background on the wines; we tasted them all blind, ordered randomly, and gave each both a relative ranking (on a scale of 1-5, with 1 being the best) and an estimate of how much we think the wine cost (or how much we would be willing to pay for it—most people rolled the two concepts into one). There was small talk but no discussion about the wines during the tasting session; rankings, price estimates, and tasting notes were all done on an individual basis and only shared with the group at the end.



Beforehand, we had a short group discussion on our own impressions. Most members of the group expressed doubt that there would be a linear relationship between the cost of a wine and their personal liking of the wine. The more experienced drinkers in the group felt more strongly that they were likely to get more enjoyment out of expensive wines, though there also seems to be a price threshold above which all expensive wines are equally good—to some of us, anyway.

Results are summarized below. (Tasting notes are at the bottom of this post.) It should be noted that most people had a hard time guessing the varietal, and some changed their ratings after learning that it was Pinot Noir. (i.e. some of the wines would be considered poor Cabernet Sauvignons but made okay Pinots.)

Price estimates

  wine 1 wine 2 wine 3 wine 4 wine 5
Judge A $25.00 $10.00 $30.00 $5.00 $15.00
Judge B $20.00 $10.00 $30.00 $5.00 $20.00
Judge C $35.00 $10.00 $17.50 $9.00 $9.00
Judge D $30.00 $8.00 $7.00 $12.00 $16.00
Judge E $15.00 $25.00 $7.50 $15.00 $20.00
Judge F $35.00 N/D $14.00 $0.00 $8.00
Judge G $20.00 $30.00 $20.00 $20.00 $20.00
Judge H $20.00 $40.00 $17.50 $7.50 $12.50
median $22.50 $10.00 $17.50 $8.25 $15.50
mean $25.00 $19.00 $17.94 $9.19 $15.06
price ranking 1 4 2 5 3



  wine 1 wine 2 wine 3 wine 4 wine 5
Judge A 3 5 1 4 2
Judge B 1 3 4 5 2
Judge C 1 3 2 5 4
Judge D 1 4 5 3 2
Judge E 4 2 5 3 1
Judge F 1 2 3 5 4
Judge G 3 2 5 4 1
Judge H 2 1 3 5 4
total score 16 22 28 34 20
final placing 1 3 4 5 2

The overall group ranking of the wines was simply determined (in the same way as the final round of a ballroom dance competition, which is to be expected given my background) by tallying up all the place rankings and giving 1st to the wine with the lowest total, 2nd to the wine with the next-lowest total, etc.


Identities of the wines

#1: Jim Ball Pinot Noir “Booneville,” 2009. Anderson Valley. ($35, direct through lot18)
#2: Pine Ridge Pinot Noir “Forefront,” 2011. San Louis Obisbo and Monterey Counties. ($17, Artisan Wine Depot)
#3: Row Eleven Pinot Noir Vinas 3, 2008. Sonoma, Santa Barbara, Monterey. ($17, K&L)
#4: Blue Fin, 2012 ($4, Trader Joe’s)
#5: Cellar No. 8, 2011 ($7, grocery store)



I entered this event with very low expectations of my own ability either to recognize or appreciate expensive wine when blinded (though I professed from the beginning to have the utmost faith in the power of environmental cues to shape my enjoyment). I emerged unsurprised at my own performance, but impressed at the degree to which the group members’ opinions agreed, at least at the extremes. Most people ranked the most expensive wine highest, and the cheapest wine lowest. However, the middle wines were a toss-up. And the group guessed correctly on the price ranking, most expensive to least expensive, but the estimates were, in general, too average—ranging $9-25, while the real prices ranged $4-35.



I can’t speak for the rest of the Stanford Wine Society organizers, but I embrace wholeheartedly the influence of environment on my own assessment and enjoyment of the wine I drink. If you have multiple ways of increasing your enjoyment in wine, why not use them? According to the studies—and those who attended Thursday’s tasting will remember hearing me describe my tendencies in this direction—paying more for a bottle, being given a personal recommendation, receiving something as a gift, receiving approval of one’s choice, or knowing something about the backstory, all increase one’s enjoyment in a bottle of wine. To address the title of the tasting, price does matter, at the very least because the price tag itself affects a wine drinker's opinion. This can only be a bad thing if one is steadfastly attached to the idea that wine is a beverage with an interesting taste, and that nothing else about it matters or is worth knowing. Astrophysicists are sometimes asked whether their understanding of the way the universe works dampens their sense of wonder when looking up at the heavens. The answer is inevitably no; the possession of deeper knowledge only deepens the amazement that is possible. This is as true of wine as it is of the stars—why else would I be part of this club?


Appendix: Group Tasting Notes - concatenated

Wine #1: Jim Ball Pinot Noir “Booneville,” 2009
oaky, dark fruit, coffee/ typical of a not-so-high-end cab, pretty good/ smoky, red fruit, medium body, well-balanced/ coffee, banana, pepper/ jammy/ extracted, concentrated, not typical of pinot, long finish

Wine #2: Pine Ridge Pinot Noir “Forefront,” 2011
light color, candy-like, Pinot, medium body. tannic, too young?/ typical pinot, not very original but still okay to drink, good with a meal/ crisp, acidic/ purple grape, a little acetic acid/ the one that was screaming Pinot – earthiness, mushroom, cola, cherry, acidic, super-true-varietal

Wine #3: Row Eleven Pinot Noir Vinas 3, 2008
spinach, cola, mushroom, not too acidic or tannic. the taste is original/ chocolate, sweet, mushroom, earthy. smooth, medium-body/ minty/ medicinal, liquorice, horrible taste. light body. worst by a long shot/ structured! good finish, nice tannins—round, medium. surprising flavors/ mineral, raspberry, strawberry, mushroom. slightly older. unusual, but not un-nice/ strongly artificial fruit, coca-cola, ajax, slight smoky note/ mild nose, most complex, has character

Wine #4: 2012 Blue Fin, 2012
no finish at all. very fruity, though. light wine to bring to a party (not with dinner). very simple, no texture. like fruit juice/ very thin, flat, watery/ caramel/ something funny about smell—medicinal? not quite right. but not totally objectionable/ medicinal, cough syrup, sickening. terrible beyond words/ light and very fruity, thin. buttery, slightly tutti-frutti/ fruit, flat

Wine #5: Cellar No. 8, 2011
nice structure, but not very fruity. tobacco, oak. not very long finishhard to place. okay, medium/ tea, earthy, but hard to put a finger on the flavor/ earthy, vegetal, berries, nice tannins. something slightly off, but decent. Cab-like/ metallic, strawberry-cherry, drinkable. simple/ pot/tobacco, herbal, simple, straightforward cherry/ smoky, strong, oaky, smooth finish