Normality and Idiocy: Bell Curves 'n Rum Reviewers!

This is the main discussion section. Grab yer cups! All hands on deck!
Post Reply
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Normality and Idiocy: Bell Curves 'n Rum Reviewers!

Post by Capn Jimbo »

It's just life baby...

Life is full of natural phenomena: height, shoe size, and people. And some that are but one step removed: SAT test scores, bond investment return, employee performance and rum reviews. What's common to all these is how the results are distributed...

Image

You need to know two things: first, that the larger the sample ("n") the smoother the "bell" becomes, and second, that the "bell curve" represents what is called a normal distribution. This distribution is both common and expected. Deviations from it are suspect and represent abnormal influences (like rum score bias, for example. Now let's look at a typical bell curve:

Image

You'll note some interesting things. In a typical normal distribution you'll find what is called the 68-95-99 Rule. Simple. This means that the normal distribution will have 68% of the results within one measure called the standard deviation. And 95% withing two measures, and 99% within three measures.

Let's consider a Five Star rum rating system. A fair and normal distribution of reviews would be something like:

1 star: 2% of rums
2 star: 14%
3 star: 68%
4 star: 14%
5 star: 2%

Mind you, things are not so perfect in real life, especially with smaller samples, but even so the distribution will follow this pattern.

And yes, yes I can hear it now, especially from the reviewers whose distributions were abnormal and biased (funny, but all of them were biased to to higher scores, hmm). "But I chose better rums in the first place!". This, of course, is horseshit. The facts:

It doesn't matter.

Whether you picked what you considered just great rums, or just poor rums, the distribution of scores made by the same reviewer will still be normal. It's like playing darts: it doesn't matter whether you aim at the center or at the perimeter - all your shots will exhibit a normal distribution around your aiming point. Of the rums reviewed, most should get your median score, with fewer and fewer getting higher or lower scores.

It's the way it is, live with it. All the Reviewer's Reviews were carefully analyzed and honestly reported with an eye toward revealing possible bias. To get our high score a reviewer had to show a few things...

First, a normal distribution of scores. Those whose scores were biased to the high end lost points. Second, we have a short list of world class rums of the five basic styles that have garnered widespread acclaim over many years, and via many venues. Though their scores may differ slightly, we expected to find some recognition of the harmony, complexity, balance and presentation of these great rums.

Last we took particular note of which rums scored at the reviewer's extremes, as these were particulary revealing of both reviewing skill and/or bias. Nonetheless, a normal distribution was basic and primary to our scoring. All factors were reported and our own scores were applied to the reviewer. Not surprisingly...

A normal distribution, lol...
Last edited by Capn Jimbo on Fri May 17, 2013 10:32 am, edited 1 time in total.
RT
Quartermaster
Posts: 119
Joined: Mon Dec 08, 2008 2:21 pm
Location: Erie PA

Post by RT »

I would love to see you do a similar analysis of the wine rating systems found in Wine Spectator/Advocate/Enthusiast etc. An alleged 100 point scale, but with 83 representing absolute plonk! So it's not just a rum problem.

By the way, the latest WINE Spectator (June 15) has an ad for Bacardi 8 on the inside front cover and Appleton Extra on the back inside cover. I guess wine has jumped the shark!
Students of the cask, reject naught but water. -Charles Gonoud, Faust Act 2
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

About 100 point scales...

Post by Capn Jimbo »

About 100 point scales...


Scales based on "100 points" are actually pretty common in the world of ratings, and they all work about the same way:
From the Wine Spectator:
* 95-100 Classic: a great wine
* 90-94 Outstanding: a wine of superior character and style
* 85-89 Very good: a wine with special qualities
* 80-84 Good: a solid, well-made wine
* 75-79 Mediocre: a drinkable wine that may have minor flaws
* 50-74 Not recommended
This compares favorably with Michael Jackson, Dave Broom, and the BTI. Here's what you need to know:

1. No spirit receives a score of less than 50, and few score less than 74.
2. So a score of "80" falls into a low average range. The bulk of wines tasted will fall into the 80-84 band.
3. Without doing an analysis, I'd guess that the Spectator ratings will fall into a nice and reliable bell curve.

This scoring (with most scores falling between 50 and 100) is really pretty similar to the grades given in school, with the bands shown being the equivalent of A-B-C-D-E and F.

Give 'em a break...
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Post by Capn Jimbo »

...bump
Post Reply