Which Spirit Scoring and Rating System is Best?
Which spirit scoring and rating system is best is controversial to say the least, as there are literally hundreds of scoring systems, and which one to use is largely a matter of utility and personal preference. In spirits, the scoring system idea was first borrowed from wine, whose commercial business cycle began earlier than that of spirits. Complete books are being written about a single scoring system. The purpose of this presentation is to provide a general overview of systems in general, their usefulness, and decide how important it is to you as an evaluator, collector, or consumer.
How did scoring systems come to be? In their purest form, scoring systems are all about providing the consumer, with a numerical guide to quality. If consumers have a “quality” rating they can trust and weigh against store prices, they can make informed decisions on cost vs quality, otherwise known as “value.” In a world with hundreds more spirits available than ten years ago, consumers need a guide to steer them in the right direction, but how do we find a guide worthy of using as our own standard?
The original intent of scoring systems to guide the consumer, opened the door for many to become noted evaluators, trusted by the public to provide useful value-related information service. In the 1950s, the wide publicity given to the payola scandal (influential rock-and-roll record promotion for kickbacks), led to the downfall of several famous music icons, and in the bigger picture, forever linked rating systems of any kind to the possibility of prurient interests, bribes, and public distrust. True or not, rumors and accusations are common in the spirits industry, so it is always a good idea to question the integrity of those who fling around rating numbers without applying an open, defined, methodical process of how the numbers were achieved.
Some large alcohol beverage retail chains employ their rating systems to promote relatively unknown “house” brands (in which they have substantial investment, and are low-cost-high-profit products). Many different rating systems are employed by perfumers, cigar critics, beer lovers and brewers, olive oil producers, aroma essence additive manufacturers, food and beverage additives, spices, and many other products. Many of these are displayed at the point of purchase (retail shelf) to imply quality to shoppers to create “impulse” buys.
The very first scoring system was good, better, and best. Critics using a 5-point system started scoring ½ points, it became obvious to use a 10-point system, then came the 20-point system for the same reason, and finally, the 100-point system evolved.
Scoring systems can be useful if employed with purpose and uniformity to build the reputation of the honest evaluator and provide usefulness to consumers. Spirits scoring systems were first adopted from wine. The 20-point scoring system was used by most European evaluators and is still used by many. In 1976, the Judgment of Paris, competition, organized by a British wine merchant, Steve Spurrier, awakened the world to the quality of California wines as world-class and alerted the consumer to the importance of a scoring and rating scale to evaluate wines. The 20-point scale was used until Robert Parker and his newsletter The Wine Advocate introduced the 100-point scale. From Wikipedia: Robert Parker (wine critic), article reference numbers deleted, the following best explains the 100-point system:
“Parker’s 100-point rating system. One of the most influential and controversial features of Parker’s wine criticism is his 100-point rating system, which he popularized in conjunction with his friend Victor Morgenroth. Parker designed the system to counter what he believed to be confusing or inflated ratings by other wine writers—many of whom he accused of a conflict of interest, as they often had a financial interest in the wines they rated. The scale, now widely imitated in other publications (such as Wine Spectator), ranks wine on a scale from 50 to 100 points based on the wine’s color and appearance, aroma and bouquet, flavor and finish, and overall quality level or potential. Therefore, 51 rather than 100 different ratings are possible. Although some critics, such as Jancis Robinson, argue that numerical rating systems are questionable—given the subjectivity of wine tasting and the variance in scores that a wine’s age and the circumstances of tasting can cause—similar 100-point scoring systems are widely used by American reviewers. Many British reviewers, such as Jancis Robinson and Clive Coates, still prefer a 20-point system.
Retailers in North America often mark wines with Parker’s point scores, using printed cards attached to the shelves. Parker cautions buyers that they should read the tasting notes to determine whether or not the wine is made in a style they will like; he states on his website:
Scores, however, do not reveal the important facts about wine. The written commentary that accompanies the ratings is a better source of information regarding the wine’s style and personality, its relative quality vis-à-vis its peers, and its value and aging potential than any score could ever indicate. — Robert M. Parker Jr., The Wine Advocate Rating System
No scoring system is perfect, but a system that provides flexibility in scores, if applied by the same taster without prejudice, can quantify different levels of wine quality and provide the reader with one professional’s judgment. However, there can never be any substitute for your palate nor any better education than tasting the wine yourself.— Robert M. Parker Jr., The Wine Advocate Rating System
Parker argues that he scores wines on how much pleasure they give him. He, and others, have said that it is the obscurity, corruption, and other problems of the appellation system that made his consumer-oriented approach necessary.”
Controversial, detractors try to discredit the Parker 100-point system by stating that a score of 50 could be kerosene, water, or panther piss. Suffice it to say that even with numerous variations, Parker’s basic 100-point system is widely adopted throughout the spirits and wine industry today, and it is precise, with enough room within 50 points to adequately describe a wine or spirit. A few unexpected results of Parker-ization are (1) a Parker 100-point score has a fourfold effect on prices and enhances sales, (2) wine-making styles in Bordeaux split, with many adopting California-inspired techniques to bring wine to the market sooner, while many stuck resolutely to the styles that made them famous for over a century. Parker Power is not the fault of the scoring system itself.
Keep in mind that professional sensory evaluators do not usually rate by numbers, and a scoring system is not generally used. Instead, professional sensory evaluators concentrate on the aromas detected and their intensity, leaving “quality” largely up to the consumer, providing further validation that scoring systems are intended as commercially related guides to aid the consumer.
Competitions are another case entirely: Many use a scoring system that is based on 100 points, with attendant medals awarded if a spirit rating falls within a certain range. It is the range that qualifies for each level that is different among many competitions. Many more competitions do not disclose a rating and do not use a rating to assign medals, but rather depend on the consensus of the judging panel. Nearly all competitions use proprietary score sheets, and most are not willing to share their methods with others. How this can affect ratings is up to you as a consumer and potential evaluator.
Gold Outstanding (98-100) An outstanding spirit with a naturally exceptional balance, complexity, and power. An example that immediately sets itself apart from others in the category.
Gold (95-100) An excellent spirit with an exceptional balance and rare and complex flavors. An example that stands out against its peers.
Silver (90-94) An accomplished spirit with good balance, strong character, and good complexity. A classic example of its style or variety.
Bronze (85-89) A perfectly well-made spirit that provides an enjoyable drinking experience. A good reflection of the category style
From the USA Spirits ratings, judging criteria are described as follows: To win a USA Spirits Ratings award you will need to score highly in all three areas of our judging criteria. That is why the USA Spirits Ratings will be decided based on the following judging formula: Q (Quality Score) + Value Score (V) + Package Score (P) = USA Spirits Ratings Score. A separate weighted score will be given for each of the three parts of the judging process (Quality gets twice the weight). The scores will be calculated with other judges for the same product on the team to give a final score out of 100 from which individual prizes will be awarded. All scores on a 100;
- Quality Score: will be marked out of 100
- Value Score: will be marked out of 100
- Package Score: will be marked out of 100
Gold: to receive a Gold medal you need to score 90 or over. Silver: you need to score between 80-89. Bronze: you need to score between 70-79.
WSET, quickly becoming the respected, leading educator and provider of certification for wine and spirits evaluators has a thorough approach to tasting wine and spirits, and their Candidate Assessment Guide Part 2: Tasting, is enlightening as to how they approach wine and spirits diagnostics. Much can be learned from their methodical, well-prepared procedures. As previously mentioned in other presentations, they also have developed an accepted vocabulary (lexicon) of terminology to accurately describe the aroma and flavor characteristics of both wines and spirits, definitely a necessity in thorough evaluation.
The publication Drink Me, provides the following description: “Scoring systems, specifically the 100-point system, date back to the 1980s. It’s important to remember, though, that scoring is used to determine where a wine sits qualitatively within a group of similar wines, thus scores are not indicative of style or palate. Scoring reshaped the way wine is viewed and how it is bought. While wine and beer evaluations are rather honest, the scoring system on the spirits front is younger in its development since evaluators are qualified individuals who possess the skills and knowledge to review the products leading some to believe the spirits side needs some more transparency.”
Different score sheets weigh different aspects of the sensory experience very differently. Within a single competition event, there may be a score sheet that is different for each general spirit type. To illustrate score sheet diversity a few examples of the more simplistic sheets can be found in Google Images and Pinterest.
The scoring of spirits is in its infancy and is nearly as diverse as the number of spirits judging events themselves. As scoring and rating evolve, procedures will change and evolve, as will vocabularies. In this presentation, we have endeavored to discuss a few ideas for consideration by (1) those who depend on the numbers to make buying decisions and (2) those who would be evaluators with a systematic approach to organizing their ratings and evaluations. Maturity and time will perfect the rating and scoring systems, and so far, we believe, as do many others, that WSET has the best handle on achieving a useful and informative diagnosis which is well on its way to becoming the new standard in wine and spirits ratings.
It seems that most bloggers, critics, and authors don’t use a scoring system, and simply write about the attributes and faults of the spirit without a numerical rating. Since these ratings are not dependent upon a numerical score, they are not necessarily held to the same rigor when evaluated, and the lack of a numerical score leaves much to opinion and personal taste. Some may find evaluations of this type to be useful, but if you want to make a business or a career of spirits evaluations, either adopting or devising a useful system with rules that you can understand and live by will be the key to consistency. If ratings are a serious consideration for your spirits purchases, be very careful who you listen to and accept as your ratings “guru.”
The important takeaway as an evaluator: (1) learn how different systems work, (2) decide whether you want to use a scoring system, and (3) determine which system fits what you would like to accomplish. Using a scoring system can add discipline and consistency to your evaluation endeavors. Otherwise, you are just another opinionated blogger.
Consumer takeaways: Don’t fall into the trap of “the higher the number the better the spirit.” Many rating systems lack a rational approach and may be only a gross number assignment without a methodical process to arrive at that number. Research and investigate your sources carefully, follow those who you believe have a solid methodology and rationale for rating, and never fail to develop your sense of rated quality vs. price whether you are a collector or just want to enhance your understanding. Don’t be a label worshipper, and realize that scarcity and low-volume production do not necessarily make a great spirit. You be the judge, rational, fair, and unbiased, using your developed sensory abilities to evaluate.
Takeaway for the future of evaluation: There is no easy way to knowledge. Read, learn, practice, or fugaboudit and become (1) a target for those who make money from ignorance and laziness, or (2) a wannabee name dropper and label worshipper. I like it or I don’t like it is not evaluation. Shortly, the spirit authors, critics, and bloggers popular today who do not use a scoring system and do not understand simple science will disappear, replaced by those who can integrate the entire spectrum of disciplined evaluation and science to provide a truly meaningful guide to the consumer.
Which NEAT glass is best for you?
Our next presentation will discuss discovering flaws in spirits.
