Subject: Digest for the period 6/1/2006 - 6/2/2006 Date: Fri, 02 Jun 2006 01:03:55 -0400 Table of contents ---------------------------------------------------------------------- 1. Judging commercial beers (beerking`at`adelphia.net) 2. subjective flaws in judging (Nathaniel Lansing) ---------------------------------------------------------------------- From: beerking`at`adelphia.net Date: Thu, 1 Jun 2006 0:17:07 -0700 Subject: Judging commercial beers I have been reading with great interest the various threads regarding judging, point assignments, and related subjects. Lots to think about, and much wisdom out there. There is one point that has been mentioned, but avoided in replies: judging of commercial beers, and the listed stylistic "commercial examples" in par ticular. It has been suggested that a good self-check/standardization would be to judge these beers yourself. It has also been suggested that these be ers should score 50 points (I am extrapolating from one comment regarding SNPA to all "commercial examples," which I do not feel is a stretch given the context of the particular comment). I don't think that was ever the intent of any of the esteemed authors of any of the style guidelines we have used over the years. The case of SNPA is a good one. Referring to category 10A, American Pale Ale, one finds a list of 9 different beers. To suggest that all 9 of these beers (Sierra Nevada Pale Ale, Stone Pale Ale, Great Lakes Burning River Pale Ale, Full Sail Pale Ale, Three Floyds X-Tra Pale Ale, Anderson Valley Poleeko Gold Pale Ale, Left Hand Brewing Jackmanâ=80™s Pale Ale, Pyramid Pale Ale, Deschutes Mirror Pond) all meet the same score (whether 50 or some other number) is nonsense. Taking this a step further, look at any issue of Zymurgy, and check out the "Commercial Calibration" column. for those not familiar, in this column, 4 of our hardest working, best known, and most knowledgeable judges (Dave Ho useman, Beth Zangari, Scott Bickham and Gordon Strong) do a judging of 2 commercial beers, complete with points and comments, per the BJCP scoresheet. The copy I have by my side right now is from Sept/Oct 2005, and the judges tackled Boston Brewing's Samuel Adams Boston Lager and Rogue Ale's Shakespeare Stout, and these two serve as reasonable examples of my point, especially since Shakespeare is listed as a commercial example of category 13E, American Stout, in which it was judged for this issue of Zymurgy. The Sam Adams scored an average of 35.5, with a spread of 42 to 32 between judges, and the Stout scored an average of 38.25, with a spread of 40 to 36 between judges. Both of these are beers many of use would regularly order, and one is listed as a commercial example, yet neither one was able to break the 40 point threshold. This suggests two things to me: 1. Recommending judges sit down and judge one of the "commercial examples," shooting for a 50 as an educational exercise is misleading, especially to newer judges. 2. Reaching that elusive, magic, 50 point score is extremely difficult, and requires a genuinely "wonderful" beer. Perhaps this also suggests that we are indeed to hard on the beers we judge. I don't feel that way, but can see someone's point if they do. I have been a judge for 20 years now, and in all that time (having judged numerous NHC sessions) I have only given one 50. It was during the second round of the NHC, and the beer went on to BOS/Homebrewer of the Year that particular year, so I feel justified in the score assigned. I was sorely tempted to take off one point, "just because." and commented to the brewer that I almost took off a point because I did not have enough of it. Thanks for the bandwidth. -- Lyle C. Brown beerking`at`adelphia.net Kuwait's only beer judge! ---------------------------------------------------------------------- From: Nathaniel Lansing Date: Thu, 1 Jun 2006 10:29:24 -0400 Subject: subjective flaws in judging It is interesting to find that, in grading the tasting portion of a BJCP examinee, consensus scores are used. I will present a testing scenario, all hypothetical but will point to the problems inherent in "judging right." A doppel-bock is presented as a taste-test beer. Two proctors grade the test beer, judge A says too hoppy, grades 34-he prefers the maltiness of Celebrator. Judge B grades 43, he finds Celebrator too cloying and prefers Salvator. So they haggle a bit and find the concensus of 37. The examinee rates the test beer a 42, knowing full well there is a range of acceptability in style descriptions. He takes a hit on grading because we dare not rate beers too highly. Too many questions come up in the scenario. Should the graders see the pre-concensus sheets? Should we score upward when we reach concensus? Score downward when reaching concensus? I know everyone wants to 'stick to their guns' so averaging will probably prevail. This is just to point out that from the very start there are going to be flaws, no, maybe only foibles, in the judging process. How can I possibly hear a national judge say that I should allow for th e water when I ding a Helles for the twangy, lingering sulphate finish? or the master that thought his Scotch ale should've taken BOS without tasting the other 245 beers in the competition? Subjectivity is built into the organism. We really can't train or talk it out. The best of us are at the mercy of our likes dislikes. ---------------------------------------------------------------------- ***** Important Subscriber Information ***** To post a message to JudgeNet, send it to judge`at`synchro.com. Send plain text only, no HTML, MIME, encoded text or attachments. Make sure you use a meaningful subject. Quote only as much material as is needed for context. To manage your subscription, go to http://synchro.com/judge/subscriptions.html or send an email to judge-request`at`synchro.com with the subject: help judge. JudgeNet is also available as an NNTP newsgroup, go to news://news.synchro.com/synchro.judge