September 15 2011
I’m aware that there are at least three strata of consumers who use wine reviews (and likely many more).
1) People that calibrate their palate to that of a critic so they can make very informed purchase decisions. These people are few and probably most closely aligned with Robert Parker or niche critics like Allen Meadows of Burghound or Charlie Olken of the Connoisseurs’ Guide to California Wine.
2) The broad swath of consumers who use scores, perhaps with some deference to the score-giver, to make retail purchase decisions. With these folks, all things considered equal while balanced against price, a 91 is better than an 88 so they go with the higher score on the shelf-talker.
3) Online armchair wine researchers are an emerging category of users. Searching for a wine presents a sort of blotter file like the dreaded “permanent record” of school days gone by. Consumers use search to research wines, validate a thought, sway indecision and incent action, sometimes in conjunction with #2.
This is linked, but separate from a recent working study presented under the banner of the American Association of Wine Economists called, “The Buyer’s Dilemma – Whose Rating Should a Wine Drinker Pay Attention to?” For a well-considered post on this topic, see Joe Roberts post at 1WineDude.
For my part, I’ve done very little wine reviewing on this site preferring instead to make any specific wine the context for bigger ideas or points I want to make (no pun intended). However, as I’ve gotten into the groove with my Forbes.com column, where there is a much broader audience, a wine-of-the-week column does have merit and I’ve started reviewing wines with more regularity.
Doing so is fun, but the most that I hope for is to be a part of the permanent record as noted in item #3. I certainly don’t have visions or a desire for anything more, but just the same, doing any sort of reviewing does open a can of worms, particularly in the case of the 2009 Red Car “Trolley” Sonoma Coast Pinot Noir, a wine that I recently reviewed and gave four stars to – which equates to a generalized “90-94” score. I don’t give precise numeric ratings. If I had to, I would have given the Red Car a 92, I liked the wine – it was earthy, nuanced, layered, balanced and it required some thought to figure out, all hallmarks of a good wine.
So, consider me SHOCKED when I saw the Wine Spectator review for this very same wine and Jim Laube gave it an 81. I was less shocked, but slightly curious when I saw that Steve Heimoff at Wine Enthusiast gave it an 86 and Stephen Tanzer gave it an 88.
Can you imagine somebody searching online for the Red Car and seeing search results that present a disparate spread along the lines of Spectator’s 81, Heimoff’s 86, Stephen Tanzer’s 88, CellarTracker’s average score of 89 and a score under the Forbes masthead of 90-94?
It would be a real WTF moment that creates more confusion instead of the consumers desired order.
This disparity in scores brings me to my point, which is the point of the Wine Economist working paper – whose score should you listen to? Well, Joe Roberts, rightfully so, says listen to your own palate. However, with the preponderance of existing and emerging wine reviewers out there, combined with an ever burgeoning tsunami of information about wine online, that’s easier said than done. The real need is for meta-aggregation of scores, a sort of super wine review database.
Neil Monnens and his Wine BlueBook represents this on some level with his monthly newsletter that aggregates wine scores for individual wines from three or more critical scores giving it a QPR rating, but this is just the tip of the iceberg compared to where information is going.
Methinks that if a stats wonk can assign a Quarterback rating to NFL quarterbacks, and Sagarin ratings for college football, there has to be a way to create a meta-rating database based on regression analysis that accounts for palate preferences across a wide diversity of reviewers to create a super score for a wine that acts as the ultimate arbiter. And I won’t be surprised if, in the near future, this emerges.
Ultimately, the ongoing debate about wine scores is for naught. The horse has already left the barn. A better conversation might be around shaping the future and the fact that the best answer to, “Whose Rating Should a Wine Drinker Pay Attention to?” might be, “Trust your palate,” but it might also be, “Tune your palate against the database.”
Posted in, Good Grape Daily: Pomace & Lees. Permalink | Comments (48) |
We’ve been talking at home for a while about a cross referenced database of reviewers scores, as well as a weighting structure that can be aligned for taste/reviewer preference. I spend enough time looking at data all day though, that I thought that this kind of project might be of interest to Paul Mabray and the Vintank crew. Interesting idea, now that technology can allow us to crunch like never before.
However the pragmatic in me, asks if it would be worth it, or if it would only further give credence that the scores from the ‘few’ are of great importance. I’ve had too many good wines, due to my own research and adventure, that the big name reviews would never touch…which is OK, since much of what they review, is stuff I could never touch.
If someone pulls this off though, they should certainly charge a subscription fee..
A wide disparity of opinion is pretty normal in movie reviews, so why not in wine reviews? I think the two are similar.
You’ve already stated the obvious answer to this question and that is “trust your palate”, but that is only half the answer. The other half is “your palate WILL change” and that is perfectly normal. A wine you love today, you might not like in the future and vice versa.
Were you REALLY SHOCKED that James Laube gave the Red Car an 81? What is interesting is that for those who really get into wine, they usually leap frog the ‘critics’ in terms of their own palate. It is an amazingly liberating day when you realize you’ve been listening to all the wrong people.
For the love of GOD, do not calibrate your palate to the critics.
Lastly, unless your buying wine for an investment, these magazines offer you very little in terms of wine knowledge as it applies to your own palate and wine experiences. Attend wine tastings in your area, if you want stories, buy a variety of books, travel to wine regions, visit tasting rooms, ask questions, and try, try, try, try, different wines from around the world. Expand your palate through your own experiences, not someone else’s.
I knew it would come this: before buying a bottle of wine, refer to your iPod wine score aggregation app; then, with all your might, try to make a decision on your own.
i, for one, think the aggregate score is even more meaningless than a critics score, which has a small shred of meaning.
i’ve spent a lot of time wrestling with the 100-point scale, and one thing i’ve noticed is that critics score wines based on what they like. so a 93-point zinfandel in wine spectator is going to be fruity and jammy, while a 93-point zinfandel in wine & spirits will be brighter and lighter.
these are already gross genrealizations. if you then take these wines out of context, and call them an 91-point aggregate zinfandel, what does that tell you?
Thanks for the comments, all.
Gab - you bring up a really good point. However, I’m no statistician, in fact I’m quite bad at math, but I do know that there are ways to create weighting variables and that sort of thing to create a critic profile which would then be weighted in the overall measurement of an aggregated score.
It’s possible. Is it doable? Probably not because of economics—an entrepreneur has to be sufficiently interested in the profit outcomes and that’s a big question mark.
Jeff
Sensory research indicates that an aggregate score would indeed be misleading. People tend to “clump” into disparate camps of varying population size when assessing flavor, due to their differing sensitivities and preferences. Camp A may prefer their Sauvignon Blancs mild and smooth while Camp B may prefer a tart and herbaceous style, and Camp C a ripe and strongly aromatic style. A single aggregate score might produce a wine that neither offends nor interests any camp. Or worse, is highly rated by one but disliked by another.
What you propose could be done on a site like CT. Just create an algorithm like Netflix uses: take your personal ratings and compare them to the database of ratings for the same wines (including the professionals that participate in CT), run a regression and get the outputs (you’ll need multiple combinations of the other reviews since there won’t be a review for every wine you’re interested in), every wine you haven’t reviewed gets automatically run through the proper equation and you’re given a predicted score for the wine, based on your own ratings of other wines.
The problem is the sheer scope of this, but Netflix has the exact same problem (not everyone has reviewed all of the same movies) so it can be done. Probably it’s done on a case-by-case basis, so take wine X. Once you click on it the system will check to see who else has reviewed wine X. Then it will cross-check everyone who has reviewed wine X with your reviews. Anyone who passes both tests gets put into the regression equation and ALL wines that you and these people have in common are compared. The system generates the equation and inputs the scores for wine X, giving you a numerical result. If you want to get really fancy you can show the result as a range, taking the error bands into consideration.
Definitely could be done.
This post reminds me of baseball…all about stats.Some folks like the camaraderie of a live game or following a team, the smell of hot dogs, the colorful pennants, collecting foul balls. Other people like to talk about stats/scores.
I used scores when starting out to see which critics palates matched my own (never occurred to me that people calibrated their palates but it explains why so many collectors defend hyped-up bottles).
Once I gained significant experience in food & wine tasting, I stopped looking at scores altogether.
I don’t see the upside to analyzing different people’s opinions about a wine, e.g. 88 vs 91 pts. I do see such a program causing more noise for consumers and headaches for wine marketers.
You have hit the nail on the head here. I think what is most important to not here is when you tasted the wine vs. when the wine was reviewed. Wine as we all know, Pinot Noir is no exception, is an ever evolving creation. One has to imaging the Red Car was tasted by the critics near or before release and the wine was still behind the clouds. My assumption is you tasted the wine a little after the reviewers and the wine began to shine, which in my experience can happen overnight. My biggest concern is the CellarTracker aggregated review, sheep? Good on you for bringing light to this important topic.
a lot of good ideas on this thread.
i like phil’s idea of creating a list of ‘things you might like’ based on your preferences, like netflix. i firmly believe that calibrating a scale to your preferences makes much more sense than creating a grand aggregate score based on how well a wine does on various 100-point scales.
i also appreciate mitch mentioning that a score is a photograph…a moment in time…and not a complete 3-dimensional understanding of a wine. as someone who has sat-in on critical evaluations with professional wine critics, i can assure you that very few wines are given the study they deserve. most wines are sniffed by a critic for a couple seconds, and if he smells something interesting, he will evaluate further. it is very easy for a critic to give a fantastic wine a quick 87 and keep moving down the line.
but, finally, as someone who has worked in wine production, i must admit there is true pride in receiving a score in the mid 90’s. i hate the 100-point scale, and there are certain critics that i think have flat-out bad taste. but there are a couple wines that i’ve worked on that i just knew were great. seeing them get a 92- or 93- is really validating. so as much as i hate the 100-point scale, i appreciate it in some strange way.
Raheja Builder Gurgaon launched luxury residential Project Raheja revanta Sector 78 in unique design with peaceful city life atmosphere. raheja gurgaon
One has to imaging the Red Car was tasted by the critics near or before release and the wine was still behind the clouds.
I am also conscious like you guys about that at least three strata of consumers who use wine reviews. I have been long time fan of Neil though his novel Wine BlueBook can’t surprise me much more. Thanks! 
sneeze guards
A single aggregate score might produce a wine that neither offends nor interests any camp. Or worse, is highly rated by one but disliked by another. Which will cause instability.
I will really appreciate the writer’s choice for choosing this excellent article appropriate to my matter.Here is deep description about the article matter which helped me more.
I am a new user of this site so here i saw multiple articles and posts posted by this site,I curious more interest in some of them hope you will give more information on this topics in your next articles.
Useful information shared..I am very happy to read this article..Thanks for giving us nice info. Fantastic walk-through. I appreciate this post..
A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one.
I am a fresh user of this site so here i saw multiple articles and posts posted by this site,I concerned more interest in some of them hope you will give more information on this topics in your next articles.
Glad to be a guest of your blog, I seem to be forward to more reliable articles and I believe we all wish to thank so many good articles, blog to share with us.
Thank you for such a fantastic blog. Where else could anyone get that kind of info written in such a perfect way?
The list you provide for best doctors is really awesome!Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic.
I wonder what difference does those three categories have.
Car Wreck Attorney
Forbes is indeed a good outlet when you want to showcase something.
fiberglass pools ky
I really enjoyed to read this article. It is a great article and i am very impressed with this information. I would like to thank you for sharing such a useful information.
I have been seeking information on this topic for the past few hours and found your post to be well written and has solid information.
This article has some great and useful information about this subject. Thank you for sharing it in an easy to read and understandable format.
This post has helped me to have another perspective. I am researching this topic for a paper I am writing. Your article provided me great insight of my topic.
It’s indeed more relaxing and enjoyable if you have music playing while drinking your wine.
As what they say, music is the food for our souls.
more enthusiastic.Music website NME.com said the track “Behind the Mask” was “an absolute revelation” on which “Jackson howls a solid-gold melody at
goodgrape with service of backlinks
Quarterback rating to NFL quarterbacks, and Sagarin ratings for college football, there has to be a way to create a meta-rating database based on regression analysis that accounts for palate preferences across a wide diversity of reviewers.
To satisfy both classic and new fans, the game’s character customization includes uniforms from the original show and films as well as the reboot,
!Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic.
It’s also partnered with two prominent Oregon growers, Goschie Farms and Coleman
Useful data. I’m really very happy to browse this kind of terminology and wording. Cheers with regard to offering persons this kind of simple data. Great walk-through. I like this kind of distribute.
Your post will be rather good and I’m sure some will find it interesting because it’s about a topic that’s as widely discussed as others. Some may even find it useful.Thanks so much for your post.
De kommer fra ledende stillinger i erhvervslivet eller den kommunale verden og får betydeligt højere løn, end de kunne få hos den humanitære organisation.
The problem right now is that there’s so little signal that a little bit of noise can cause problems.
This summer the President & CEO here at the Alliance for Aging Research, Dan Perry, was asked to participate in the annual melding of the foremost medical minds known as dhea, lipo 6 unlimited and melatonina comprar
I am really impressed from your immaginative thinking and impressive stated information, i am glad.
When I first visit this site and try to look what this site talks about I was really amazed with.WOW!fantastic.You could have get new important and interesting topic to be discuss. Really feel great that I visit this site.
Great beat ! I would like to apprentice while you amend your website, how can i subscribe for a blog website? The account helped me a acceptable deal. I had been a little bit acquainted of this your broadcast offered bright clear idea
But i have never met a guy who used to read reviews about any wine before taking. All of them just go to the shop and asked for and then go home and take simply.
I have been in look for of some information about it almost three time. You served me a lot indeed and analyzing this your publish I have found many new and useful information about this topic.
This work is licensed under a Creative Commons Attribution-No Derivative Works 3.0 Unported License
Design by The Blog Studio
Print
Thank you for the kind words, bruthah!
That score disparity for the Red Car doesn’t surprise me at all. It’s more kindle for the fire that alights the torch I’ve been waving for a couple of years now, I suppose. Wow, that is one really bad image, but screw it, let’s go with it!
People need to stop being wuss-bags & taking the lazy way out in assuming that the ratings for products are all the same because the scale used is identical.
No one in the right mind would buy a car that Consumer Reports rated highly JUST because it got that rating and WITHOUT bringing their own preferences into play in the purchasing decision. With wine, knowing your palate preferences is like being able to take a bit of a test drive without even opening the bottle, and what kind of loon would buy a car without a test drive?
Seriously, people - muster up the iota of courage that it takes to learn some thing about yourself that will help you navigate this wine craziness with relative ease (or at least the minimized chance of additional headaches)!