Reviewer's Review: Chris Carlsson's Spirits Reviews

This is the main discussion section. Grab yer cups! All hands on deck!
Post Reply

How do you rate Chris Carlsson's Spirits Reviews (5 is best)?

5
0
No votes
4
0
No votes
3
0
No votes
2
1
100%
1
0
No votes
 
Total votes: 1

User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Reviewer's Review: Chris Carlsson's Spirits Reviews

Post by Capn Jimbo »

The Quotable Chris Carlsson...

You know what's sad? Simply put, (a) that there are so very few rum review sites and (b) that most of those are commercially influenced. The nice part about the real noble spirits - wine and whisky - is that in comparison, there are literally hundreds of websites, led by a number of well known, credible and honest expert reviewers. Think Michael Jackson, Dave Broom, Jim Murray, F. Paul Pacult and Rober Parker. As for rum?

Crickets.

Chris Carlsson joins this small group with yet another marginal reviewing site. Surprisingly a few distillers actually cite Chris, along with some of the faux "awards" given by some of the fee-fie-faux-flim-flam rum "competitions". Let's see why...

Image

Like the Artic Wolfboy and most of the others, Carlsson's reviews are almost exclusively weighted toward the top scores. In his 10 point system, "5" is considered average. Of the 47 rums "reviewed" only one - yes one - rum is considered "average", with just one other lone rum as "below average".

To be credible, most of the reviews should be average, bracketed by ever fewer lesser or greater rums. Not here.

His actual average score is an "8" - the equivalent of 4 stars, "highly recommended", in the usual 5-star system. That only two of forty six rums are average or less is simply not credible. Carlsson's reviews are softballs and obviously designed to garner free product. Although he claims he's not influenced by freebies, the analysis tells a different story.

He simply loves to hand out "8's", even to the few truly world class rums he's reviewed. And just like his cohorts who wouldn't know an altered rum if they fell into the vat, he has the typical sweet bias - with the syrupy sweet Diplomatico's garnering from "9" to "10". And like them too, he exhibits the usual age and price bias (favoring older/more expensive).

This, my fellow idiots, is not reviewing; rather, it is pandering. It's fun! You get free product, you hand out "8's", you get quoted, you get free product. It's not all bad. He nails a few rums - including Prichards, Mount Gay, Don Q Gran Anejo and the Appletons - but they still earn his usual "8".

And so it goes.


Score (ten is best): 4 - for a few accurate descriptions.
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Carlsson responds...

Post by Capn Jimbo »

Carlsson responds...

I like Chris, and accordingly will post his response:
Carlsson: "First, Obviously you didn't read the Bacardi reviews before writing about me that would have showed I don't hesitate to trash bad spirits of any kind. Or maybe you did and you are taking Bacardi's money and decided to trash me because I disliked their rum so much - something you completely forgot to mention in your statistical review of me. Interesting in both the fact you did not use it and the timing."
Hmm, do I hear paranoia here? Maybe I don't like Chris, lol. Nah, I still do. But it's clear that Chris knows neither the meaning of "normal distribution", nor my feelings regarding Bacardi (whose politics and products I have almost unanimously and repeatedly criticised).

I won't waste your time with my entire email response, but suffice it to say that I respectfully responded and tried to illuminate the night for dear Chris. Here's an exerpt:
Compleat Idiot: "Chris, it's really about overall scoring. You state your average score is "5", yet only one rum gets that score. Your actual average score - for all reviews - is 7.9, call it 8. That is imbalanced, and here's why...

It's all about what's called a "normal distribution". It doesn't matter what kind of rums you choose, over time your scores will cluster about a midpoint and produce what is called a "bell curve" or normal distribution..."
I proceeded to suggest he redo his scores and consider a five star system with most rums scoring three. Stay tuned for his response, if any...
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Carlsson Rescored...

Post by Capn Jimbo »

Carlsson Rescored...

While I await Chris' response to my good natured advice (to rescore his reviews), I thought it would be instructional to simply do it for him...

Image

Nice, eh?

Now this is a normal distribution, and it was really quite easy to do. Simply, his 10's became 5 stars, his 9's became 4 stars, his 8's became 3 stars (average), his 7's became 2 stars, and...

All the rums scored from 1 through 6 became 1 star rums (this because so few rums received these low scores). This proves two things:

1. That Chris' scores for all but three of his forty seven reviews were quite high.

2. That achieving a credible normal distribution was actually pretty easy, assuming Chris has the cajones to downgrade his scores.

This would settle the matter, save for one thing. He'd have to call some of the world's leading rums simply "average". Which they are not, he likely won't, and that's that.

Good luck Chris...
User avatar
Capn Jimbo
Rum Evangelisti and Compleat Idiot
Posts: 3550
Joined: Mon Dec 11, 2006 3:53 pm
Location: Paradise: Fort Lauderdale of course...
Contact:

Chris proved me wrong...

Post by Capn Jimbo »

Chris proved me wrong...


And I applaud him. After considering my suggestions, I got the following note from Chris:
Carlsson:
"Hi, Thanks, will do (on both counts)!

I admit the rating system needs work because as you will notice I almost never give anything a 10 and especially a 10/10. Also I think maybe I'm thinking a little too much like school grades (80-C Average, 70-D Passing, etc.,).

The value/price is also another complicating factor because it is a separate point system- as you know I might score a expensive spirit high in the overall quality system, but if I think it is too expensive for what it is it gets a serious downgrade in that area. As my mother in law used to say, anyone can find a nice dress for $200, the hard thing to do is find a nice dress for $50"

Chris
It takes a real man - or a Compleat Idiot - to admit he's wrong, and never more than in this case where one's rating system is at stake. As critics we work very hard to be fair, but also to be honest.

Some of our targets are left speechless, others respond with nasty emails. But only Chris saw the truth and value of the analysis and has decided to revamp his entire website and rating system.
Carlsson: "Hi, Yes we are going to revamp the scoring, I just have to decide exactly how to carry it out- otherwise we have 600 + pages to rcscore by hand and adjust graphics. It's going to take a while.
Cheers,
Chris"

That takes cajones! I am in awe of this man and his dedication to fair reviewing. The Wolfboy should take note...
Post Reply