Which Spirits Evaluation and Rating System is Better?
Which Spirits Evaluation and Rating System is Better?
Which spirit evaluation and rating system is better is controversial to say the least. There are literally hundreds of rating systems. Which one to use is largely a matter of utility, personal preference, and accuracy. The earlier commercial business cycle of wine was the birthplace of the rating system. Spirits rating systems evolved from wine.
Complete books are written about a single scoring system. The purpose of this presentation is to provide a general overview of systems in general, their usefulness. You decide how important it is to you as an evaluator, collector, or consumer.
How did Spirits Evaluation and Rating Systems Evolve?
In their purest form, rating systems are all about providing the consumer, with a numerical guide to quality. If consumers have a “quality” rating they can trust and weigh against store prices. Additionally, they can make informed decisions on cost vs quality. In a world with hundreds more spirits available than ten years ago, consumers need a suitable, standardized system.
The original intent of rating, evaluation and scoring systems was to guide the consumer. They opened the door for novices to become noted evaluators, trusted to provide useful value-related information. In the 1950s, the payola scandal (influential rock-and-roll record promotion kickbacks) led to the downfall of several famous music icons. The public has linked rating systems to the possibility of prurient interests, bribes, and public distrust.
True or not, rumors and accusations are common in the spirits industry. Therefore, one should question the integrity of those who fling around rating numbers without explanation. Evaluators should clearly define and methodically explain the rating process, including how they calculate the numerical scores.
Rating Practices can Depend on an Internal Store Marketing System
Some large alcohol beverage retail chains employ their own rating systems to promote relatively unknown “house” brands. These are brands in which they have substantial investment, are low-cost-high-profit. Perfumers, cigar critics, beer enthusiasts, brewers, olive oil producers, aroma and flavoring manufacturers, spice producers, and many other industries use a wide range of rating systems. Retailers often display these ratings at the point of sale to signal quality and encourage impulse purchases.
The First Rating System was Good, Better, Best
Critics using a 5-point system started scoring ½ points. Obviously, a 10-point system was needed, then the 20-point system. Finally, the 100-point system evolved.
When evaluators apply rating systems consistently and with a clear purpose, they can build credibility and provide useful guidance to consumers. Spirits evaluators originally adapted their scoring systems from wine evaluation. Most European evaluators traditionally used the 20-point system, and many continue to use it today.
Judgement of Paris Used a 20-Point Rating System
In 1976, British wine merchant Steven Spurrier organized the Judgment of Paris, which established California wines as world-class competitors and drew consumer attention to systematic wine evaluation. Evaluators commonly used the 20-point scale until Robert Parker popularized the 100-point scale through The Wine Advocate.
Enter the Robert Parker Rating System
From Wikipedia: Robert Parker (wine critic), article reference numbers deleted, the following best explains the 100-point system:
“Parker’s 100-point rating system. One of the most influential and controversial features of Parker’s wine criticism is his 100-point rating system. Parker invented it with his friend Victor Morgenroth. He designed the system to counter what he believed to be confusing or inflated ratings by other wine writers. Eventually, he accused many of a conflict of interest, as they often had financial interest in wines they rated.
The scale is now widely imitated in other publications (such as Wine Spectator). The rating system scores wine from 50 to 100 points by evaluating its color, appearance, aroma, bouquet, flavor, finish, overall quality, and potential. Because the scale begins at 50, it provides 51 possible scores rather than 100. Critics such as Jancis Robinson question numerical rating systems because wine tasting involves subjective judgments, and a wine’s age and tasting conditions can influence its score. Nevertheless, American reviewers widely use similar 100-point systems. Jancis Robinson and Clive Coates, still prefer a 20-point system.
Wider Adoption of the Parker Rating System
North American retailers often display Parker’s wine scores on printed shelf cards. However, Parker advises buyers to read the accompanying tasting notes to determine whether a wine’s style matches their preferences. He states on his website:
Scores, however, do not reveal the important facts about wine. The written commentary that accompanies the ratings is a better source of information regarding the wine’s style and personality, its relative quality vis-à-vis its peers, and its value and aging potential than any score could ever indicate. — Robert M. Parker Jr., The Wine Advocate Rating System
No scoring system is perfect, but a system that provides flexibility in scores, if applied by the same taster without prejudice, can quantify different levels of wine quality and provide the reader with one professional’s judgment. However, there can never be any substitute for your palate nor any better education than tasting the wine yourself.— Robert M. Parker Jr., The Wine Advocate Rating System
Parker argues that he scores wines on how much pleasure they give him. He, and others, have said that it is the obscurity, corruption, and other problems of the appellation system that made his consumer-oriented approach necessary.”
Delving into the Parker System
Controversial, detractors try to discredit the Parker 100-point system by stating that a score of 50 could be kerosene, water, or panther piss. However, Parker’s basic 100-point system is widely adopted throughout the spirits and wine industry today. It is precise, with enough room within 50 points to adequately describe a wine or spirit.
A few unexpected results of Parker-ization are (1) a Parker 100-point score has a fourfold effect on prices and enhances sales, (2) wine-making styles in Bordeaux split, with many adopting California-inspired techniques to bring wine to the market sooner. However, many stuck to the styles that made them famous for over a century. Parker Power is not the fault of the scoring system itself.
Professional sensory evaluators generally avoid numerical ratings. Instead, they identify aromas and measure their intensity, leaving consumers to judge overall quality according to personal preference. This distinction shows that scoring systems primarily serve as commercial guides for consumers rather than objective instruments of sensory analysis.
Competitions are Another Case Entirely
Many use a scoring system that is based on 100 points, with attendant medals awarded if a spirit rating falls within a certain range. It is the range that qualifies for each level that is different among many competitions. Many more competitions do not disclose a rating and do not use a rating to assign medals, but rather depend on the consensus of the judging panel. Nearly all competitions use proprietary score sheets, and most are not willing to share their methods with others. How this can affect ratings is up to you as a consumer and potential evaluator.
The IWSC Spirits Rating and Evaluation System
Gold Outstanding (98-100) An outstanding spirit with a naturally exceptional balance, complexity, and power. An example that immediately sets itself apart from others in the category.
Gold (95-100) An excellent spirit with an exceptional balance and rare and complex flavors. An example that stands out against its peers.
Silver (90-94) An accomplished spirit with good balance, strong character, and good complexity. A classic example of its style or variety.
Bronze (85-89) A perfectly well-made spirit that provides an enjoyable drinking experience. A good reflection of the category style
The USA Spirits Ratings
From the USA Spirits ratings, judging criteria are described as follows: To win a USA Spirits Ratings award you will need to score highly in all three areas of our judging criteria. USA Spirits Ratings will be decided based on the following judging formula: Quality Score (Q) + Value Score (V) + Package Score (P) = USA Spirits Ratings Score. A separate weighted score will be given for each of the three parts of the judging process (Quality gets twice the weight). Scores are calculated with other team judges for the same product. Finally, a score out of 100 will determine prize awards. All scores on a 100;
- Quality Score is marked out of 100
- Value Score is marked out of 100
- Package Score is marked out of 100
Gold: to receive a Gold medal you need to score 90 or over. Silver: you need to score between 80-89. Bronze: you need to score between 70-79.
The WSET Spirits Rating System
WSET has become a leading provider of wine and spirits education and certification and applies a thorough, systematic approach to tasting. Its Candidate Assessment Guide, Part 2: Tasting explains how WSET approaches wine and spirits evaluation and offers valuable lessons through its methodical, carefully structured procedures. WSET has also developed a widely accepted lexicon that enables evaluators to describe wine and spirits aromas and flavors accurately—a necessity for thorough evaluation.
Media Notes on Spirits Rating Systems
The publication Drink Me, provides the following description: “Scoring systems, specifically the 100-point system, date back to the 1980s. Remember that evaluators use scores to rank a wine’s perceived quality among comparable wines; the scores do not describe its style or predict whether it will suit an individual palate. Scoring has reshaped how consumers perceive and purchase wine. While wine and beer evaluations are rather honest, the scoring system on the spirits front is younger in its development. Evaluators are qualified individuals who possess skills and knowledge. Some believe the spirits side needs more transparency.”
Different rating sheets weigh different aspects of the sensory experience very differently. Within a single competition event, there may be a rating sheet that is different for each general spirit type. A few examples of simplistic sheets are found in Google Images and Pinterest.
Rating Systems are Still Changing
The rating of spirits is in its infancy and is nearly as diverse as the number of spirits judging events themselves. As scoring and rating evolve, procedures will change and evolve, as will vocabularies. In this presentation, we have endeavored to discuss a few ideas for consideration by (1) those who depend on the numbers to make buying decisions and (2) those who would be evaluators with a systematic approach to organizing their ratings and evaluations.
Maturity and time will perfect the rating and scoring systems, and so far, we believe, as do many others, that WSET has the best handle on achieving a useful and informative diagnosis which is well on its way to becoming the new standard in wine and spirits ratings.
Most bloggers, critics, and authors don’t use a system, and write about the attributes and faults of the spirit without a rating. Their ratings are not rigorous, since they are not numerical. The lack of a numerical score leaves much to opinion and personal taste.
Some may find evaluations of this type to be useful, but if you want to make a business or a career of spirits evaluations, either adopting or devising a useful system with rules that you can understand and live by will be the key to consistency. If ratings are a serious consideration for your spirits purchases, be very careful who you listen to and accept as your ratings “guru.”
The important takeaway as an evaluator: (1) learn how different systems work, (2) decide whether you want to use a scoring system, and (3) determine which system fits what you would like to accomplish. Using a scoring system can add discipline and consistency to your evaluation endeavors. Otherwise, you are just another opinionated blogger.
Rating Systems: The Consumer Takeaways
Don’t fall into the trap of “the higher the number the better the spirit.” Many rating systems lack a rational approach and may be only a gross number assignment without a methodical process to arrive at that number. Research and investigate your sources carefully, follow those who you believe have a solid methodology and rationale for rating. Never fail to develop your sense of rated quality vs. price whether you are a collector or just want to enhance your understanding. Don’t be a label worshipper and realize that scarcity and low-volume production do not necessarily make a great spirit. You be the judge, rational, fair, and unbiased, using your developed sensory abilities to evaluate.
Takeaway for the future of evaluation: There is no easy way to knowledge. Read, learn, practice, or fuggedaboudit and become (1) a target for those who make money from ignorance and laziness, or (2) a wannabee name dropper and label worshipper. I like it or I don’t like it is not evaluation. Shortly, the spirit authors, critics, and bloggers popular today who do not use a scoring system and do not understand simple science will disappear, replaced by those who can integrate the entire spectrum of disciplined evaluation and science to provide a truly meaningful guide to the consumer.
In addition, the evaluation and rating glass must be designed for sensory evaluation to guarantee accurate rating and evaluation scores. NEAT is the first and only sensory designed glass ever.
