Reviewer's review: Beverage Tasting Institute

[Capn Jimbo]

Beverage Tasting Institute: The Gold Standard

Now, now I can hear the oxen owners screaming as their lead animals are being gored, and yes I'm sure you have your own personal favorites. But I'll say it again - the BTI is the gold standard. There's a number of reasons for this:

1. BTI accepts no advertising, nor sponsorship. Participation is entirely voluntary with all distillers paying the same standard price for evaluation. All products submitted are reviewed and scored, and include copious review notes. Good or bad, gulp, these are all published.

Not like safe, unpublished shill awards at the faux competitions.

2. Tasting is done by a panel of senior staff members, all of whom are recognized and experienced sensory evaluation experts, plus a group of outside experts that have undergone training in BTI's proprietary evaluation methods.

Not a bunch of promoter wannabees and hangers on.

3. All the flights are performed blind, neat, and under very closely controlled conditions using Reidel stemware. Tasting is done at the same time of day at 68 degrees in pleasant and calm testing rooms. Flights are limited to prevent tasting fatigue.

Not like the 50+ rums "tasted" by ever more looped gang of guzzlers.

4. BTI believes in the use of a shorter scale (for accuracy) and proceeds in two rounds. The first round is scored:

1--Not recommended (less than 80 points).
2--Of sound commercial quality, though not overly exciting (80-84).
3--Shows style and character, yet probably not of the highest merit (85-89).
4--Highest quality.

Spirits earning at least two "4's" (highest quality) are reevaluated in a second round:

3--Very good, but not of the highest merit (88-89 points).
4--Truly excellent in style and distinction (90-92).
5--Outstanding, though not quite one of the world’s finest (93-95).
6--Provides a world-class experience (96-100).

The final results are then translated into a modified 100 point scale:

96-100--Superlative
90-95--Exceptional
85-89--Highly Recommended
80-84--Recommended
less than 80--Not Recommended

Not broken down into disintegrated, simplistic systems or "methods", wherein the bottle and label may mean more than the rum.

5. Interestingly this system is based on use of a the "mode" (most frequent score) rather than the much more typical "average". As we all should realize, there is no such thing as an average anything. This is particularly true in idiocy, ergo I am "The Compleat Idiot" of rum. QED. Spirts which seem controversial, or earn widely disparate scores are reevaluated to eliminate "bad bottles" or the like.

6. I have evaluated the distribution of scores:



As shown the scores fall into a very reassuring bell curve (aka normal distribution), which is strong reassurance for the lack of bias, and is as it should be.

7. Last, and least important, I have closely examined my own highly rated reference rums against BTI's scoring of them. These matched up well, both in scoring and and profiles.

*******

Score (ten is best): 9 (due to some inaccessibility of reviews).

A reliable and thorough resource that has evaluated more rums, more completely than any other resource on the net. You are well advised to strongly consider any rum scoring 90 or above at BTI. For a fascinating debate regarding BTI, please be sure to click on this thread (link): the BTI debate occurs near the bottom of the page.

***

More rum reviewer's reviews:
1. Beverage Tasting Institute, for a real bell curve. The gold standard.
2. Arctic Wolf, biased, bitter reviews. I'm serious!
3. RnD Rum Reviews, biased, favoring sweet, smooth and expensive. Rum with Koolaid.

[NCyankee]

I still say I really question BTI's ratings - I mean if you cut off at 89 or 90, you would never try Pussers Blue Label which is 84. They give Zafra 95. They rate several mediocre rums higher than Pussers - Diplomatico Reserva (92), Gosling's Gold (89), Cruzan dark Aged Rum (89) , Don Q Cristal (88), Bacardi reserva and select (88 - higher than the bacardi 8 yr at 87). Geez they rate the godawful bacardi "O" the same as Pusser's at 84!

They do seem to love the white rums, giving Flro de cana 4 yr old white a 93, as well as the El Dorado 3 yr old - odd that they don't rate any of the older El Dorados ar any of the Barbancourts, yet they do rate a ton of rums I have never heard of - Elements 8, the entire range of Lamb's. And rums that no serious reviewer would probably bother with, like all the bacardi flavored crap.

They might have a nice bell curve distribution but I very much question their ability to reliably judge so many different spirits, as I find their Scotch ratings to be similarly all over the place. I also know a wine and beer store owner who formerly worked as a regional sales rep for a major wine company, who says their wine ratings are "a joke" in his own words.


*******

Capn's Log: So how do ya really feel? The notion of a bell curve is not whether BTI agrees with your personal bias, or with that of your beer store owner (a survey of one). That's no more than a matter of personal taste. You really ought to take the time to go to BTI and review their methodology, which is completely unbiased and blind, and which proceeds through layers of analysis unmatched by any other resource.

So whether you two agree or not is immaterial - the most important thing is that the results form a bell curve and a normal distribution of results - which indicate an honest process and an honest distribution, with most scores being around the median,and with progressively fewer higher and lower scores.

That is reliable.

Having passed that hurdle, it's up to you whether you agree with the tastes of their expert tasting panel or not. If you do, then BTI is your resource; if not, you need to keep looking. But in either case, your personal disagreement has no bearing on the reliability of BTI, which is impeccable.

Let's see - you disagree with BTI, so they must be unreliable. And I disagree with you, so...

[Capn Jimbo]

It's like this...


Normally I wouldn't bother as my "Capn's Log" note above should suffice, but I like this comment, as it exemplifies common misunderstandings about ratings.

First, a "bell curve" analysis reveals whether any group of data, in this case rum reviews, are scored honestly and without bias. A non-biased reviewer will have a normal distribution, with most scores near the median, and quickly dropping off for higher and lower scores. There should actually be relatively few scores at the extreme high or low end.

For more details (click here).

This has everything to do with honesty, and nothing to do with taste. Different reviewers will have differing tastes and preferences, which may not match your own. What's much more important is whether the reviews as a whole represent an honest, unbiased and normal distribution. Ask yourself...

Would you prefer a reviewer whose tastes are closer to yours but gives everything 10's? Of course not.

While there are a couple of individual reviewers whose scores are reliable, their tastes are very specific. The big, big advantages of BTI are multiple:

1. They have reviewed, by far, more rums than anyone else. Thus their reviews reflect a large and varied expert experience.

2. Their expert panels are chosen wisely and do rotate. The tastings are conducted under stringent and reproduceable conditions, and proceed in multiple blind tastings and retastings for the greatest possible accuracy and reproduceability.

Their methodology is impeccable, and represent a consensus of talented and unbiased experts. By representing a consensus - and not the opinion of an individual reviewer - the results are competent and especially enlightening.

If the Yankster doesn't agree with BTI, rather than criticising them, he'd be well advised to learn from their results, and to consider what he might be missing.


Let's respond specifically

"I mean if you cut off at 89 or 90, you would never try Pussers Blue Label which is 84."

I too disagree with their rating of Pussers, but that's really irrelevent. Based on the distribution of their scores, I stand on my comment: that you won't go wrong choosing BTI's 90 (these would be 4 or 5-star rums in a 5-star system). Still, an "84" is bordering on their "highly recommended" (think 3-star), and Pusser's 15 Year gets a "92" (think 4-star: personally I'd reverse these scores). Furthermore, I'd hope that you aren't picking rums by score alone, but have some primary reason for doing so. The reviews of both are intriguing, with relatively accurate descriptors - with the 15 Year summarized as "...somewhat mysterious aged rum that speaks of a different time" and the Blue Label as "A bit jumbled, but interesting."

Next:

"They rate several mediocre rums higher than Pussers - Diplomatico Reserva (92), Gosling's Gold (89), Cruzan dark Aged Rum (89), Don Q Cristal (88.), Bacardi reserva and select (88 - higher than the bacardi 8 yr at 87). Geez they rate the godawful bacardi "O" the same as Pusser's at 84!".

You are not being fair.

Now we're into your personal opinions regarding a number of different classifications of rum - silver, gold, dark, flavored and aged - in comparison to a Navy rum (actually THE Navy Rum). This is really quite unfair. As I have long held, rums should be compared by style, and secondarily by class. BTI divides rums into these classes and compares same against same (eg silvers against silvers). To compare these silvers, golds, flavored and darks with the aged Pussers offerings - which you just did - is comparing apples to oranges, or wines to beers. And last...

"I also know a wine and beer store owner who formerly worked as a regional sales rep for a major wine company, who says their wine ratings are "a joke" in his own words. "

This fails on its face. Sales reps have never, ever been known for actually understanding their own products, much less those of others. A survey of a wine rep who now owns a beer and wine store is not really meaningful, not only as a survey of one, and but particularly regarding rum.

The statement you have made is completely valid: you do not agree with the tastes of BTI, as exampled by your Pusser's Blue Label. Regarding Blue Label, I agree with you. And BTI disagrees with us. None of which is relevent to the normal distribution of their scores.

I'll say it again. The Reviewer's Reviews are based on three factors: primarily, are the scores fairly reported to compose a normal distribution (free of bias); secondarily, how did the reviewer report a selected number of recognized world class rums and last, what rums appeared at the extreme highs and lows.

It goes without saying that the only reviewer with whom you'll find total agreement is yourself. When you disagree with most reviewers you are disagreeing with a single person - when you disagree with BTI you are disagreeing with a consensus.


Summary

Yankster, all said, I do thank you for posting as it indirectly raises what I think are BTI's shortcomings. First, that BTI's descriptors, though accurate, often do not fall within the experience of many of the readers. And second, that BTI (like most review sites) ignores what should be the primary classification - by style: Bajan style, Jamaican style, Demeraran, Cane Juice, and Cuban. Neither of these shortcomings are fatal.

An enlightened reader can still search for rums of the style(s) he/she wants to try against others of the same style. The best source of listings by style?

Right here in this forum. When a rum receives a high score here, it is in comparison to the reference standard for each style. If that score is higher, then we might just have a new style standard. But you knew that, right?

Thanks for posting.

[Beukeboom]

Just checked out their rum ratings and feel a bit better knowing that several of the rums I really like are rated highly by them. One rum however, Charboneau rum from Natchez, MS, I don't particularly like because of some weird flavor that I cannot identify. I leave the possibility open that it may be I got a bum bottle and that their white rum is quite good. If I come across another bottle I will consider getting it to see if that is the case.

But I digress...

The BTI list gave me several rums to try to order through the local chain store in order to try.

[GregS]

I was glad to see this thread. I'm new and I've been hitting a bunch of rum rating sites to see what I should try. I want to get what is seen as being pretty good rum. But I'd find that on BTI they have something like Mount Gay XO as a 95 and then I look on Rum Ratings app and it's a 7.8. I knew that the Rum Ratings app is people rating it how they like something. So it's nice to see that BTI is a blind tasting and that all the people doing the rating are trained the same way. Makes things much more useful.

I know that I won't always like what they like, but it helps a newbie pick rum that is seen as being good.

I do think that BTI could improve their testing by breaking the ratings up to style (Jamaican with Jamaican). Also rating similar rums together., (aged with aged)

[Capn Jimbo]

If you look through the reviews of reviewers you'll find a few whose results are relatively unbiased and that exhibit an expected distribution of results.

Unfortunately there are those whose biases are notable and misleading - especially those who depend on freebies...

[GregS]

It's always annoying when someone reviews something and their bias is obvious. But I suppose it all depends on what the reviewer's goal is. If it is to look at the rum judging just how it represents the style or something like that, there can be no bias. However, if the reviewer is trying to get across his opinion, then bias is expected.

I started looking at BTI because a lot of my rum is bought at Total Wine and they reference BTI fairly often.