Subject: Digest for the period 5/24/2006 - 5/25/2006 Date: Thu, 25 May 2006 01:00:37 -0400 Table of contents ---------------------------------------------------------------------- 1. Re: Scoring, a statistical look (David Houseman) 2. Second Call for Judges and Stewards at the NHC (Don Ferris) 3. Re: Flight Size, Acounting for Points, et al (Mike Dixon) 4. Call for judges - B3 2nd Annual Forum HBC (Chris Love) 5. 90/80/70/60.... (Dr. L.A. Swihart) 6. RE: Judging (John Peed et all) (Jon Tobey) 7. Re: accounting for points (Leo Koster) ---------------------------------------------------------------------- From: David Houseman Date: Wed, 24 May 2006 07:53:00 -0400 Subject: Re: Scoring, a statistical look Spencer, A great statistical analysis. Thanks. My empirical evidence from my own judging would align with your analysis. But I've never kept track so it's only an educated guess. Let's see if we can get you the data for a very large national competition. Dave Houseman ---------------------------------------------------------------------- From: Don Ferris Date: Wed, 24 May 2006 08:40:50 -0400 Subject: Second Call for Judges and Stewards at the NHC Second Call for Judges and Stewards at the 2nd Round of the National Homebrew Competition! Held in conjunction with the AHA National Homebrew Conference in Orlando, FL 22-24 June 2006 (lots of pre-conference events schedule fore 21 June! See www.beertown.org for additional details) Come be part of the largest Homebrew Competition in the World! We still need more Judges and Stewards! The AHA would like to have ~9 judges per category. (Here’s the preferred procedure: Split the 24-27 entries in each category into three 8-9 beer flights, with three judge panels per flight. Each panel will push 2-3 beers. Then the senior judge from each initial panel will sit together and determine the top three using a fresh bottle of each beer.) Using simple math, that means there are 243 judging positions and 81 stewarding positions that need to be filled (9 judges (3 stewards) x 27 categories = 243 judging and 81 stewarding spots) just to determine the medal winners! Pre-registration for judging/stewarding is essential as we are trying to pre-assign flights to facilitate the process so don’t delay! Sessions will be held Thursday Morning, Thursday Afternoon, and Friday Morning. (Again simple math, if we do 9 categories each session then we need 9 x 9 = 81 judges per session!) Preference to judging positions will be given to people holding a BJCP certification. However, if not enough BJCP judges are available, then individuals with judging experience that do not have BJCP certification may be used. If this occurs, no more than one (1) non-BJCP certified judge will be on any three (3) judge panel, and at least one (1) National or higher ranked BJCP judge will be part of that same three (3) judge panel. If you would like to Judge or Steward please send the following information to Judges`at`flbrewer.org or stewards@flbrewer.org respectively: Last Name First Name Preferred Name or Nickname BJCP # (if applicable, even if just planning to steward) BJCP Rank (if applicable) Categories eligible to Judge or Steward Categories ineligible to Judge or Steward (i.e., you have an entry in the 2nd rd of the NHC) Categories you would prefer to Judge or Steward Categories you would not prefer to Judge or Steward Time periods you would prefer not to judge (i.e. there is a presentation you would like to attend) State you live in (Will be used to try and ensure each panel has regional diversity) Your homebrew club (if applicable, for statistical purposes to see how many different clubs are represented) email address (or other means of contacting you) Also ensure you have registered for the National Homebrew Conference to participate in other activities going on around and after the judging/stewarding. If you have already contacted us about judging or stewarding, you are “on the list” Look forward to seeing you in Orlando! Don Ferris NHC 2nd Round Judge Coordinator Judges`at`flbrewer.org Dani Parris-Exline NHC 2nd Round Steward Coordinator stewards`at`flbrewer.org Space Coast Associates for the Advancement of Zymurgy (SAAZ) www.saaz.org ---------------------------------------------------------------------- From: Mike Dixon Date: Wed, 24 May 2006 09:40:04 -0400 Subject: Re: Flight Size, Acounting for Points, et al In response to John Peed's various posts, how about taking a SNPA (or other beer), judge it on a scoresheet taking into account why you removed points and then post a link to illustrate your idea. I cannot see going on in great detail about why someone lost a point here or a point there. It is not always a single thing that caused a deduction. I'm the type of person who feels that the beer is perfect in the bottle and capped and 99% of the time if that was all I was judging the beer would get a 50. Once you open the bottle then you can begin to evaluate the beer and see if it falls within the guidelines, if it is flawed through process or sanitation, and finally how it can be improved. If it cannot be improved then it should get the 50. The further it needs improvement then, IMO, the beer should get more feedback as to how to improve. Lately I am running out of room on overall impression and could use two or so more lines. Anyone want to tweak the Beer Scoresheet? Take one line from appearance and one from Mouthfeel and add them back to Overall Impression...oh well, it was worth a shot. If I waxed poetic about why the brewer should receive an 8/10 instead of a 9/10, I don't really see it serving a purpose but to fill in white space I already do not have. In response to 15 beers in a flight being too much, perhaps, but what makes 12 more appealing, and 8 even better? The last comp I organized had a bit over 150 entries and we were able to keep the flights less than 12. I think at our recent comp we were able to keep them below 14. The problem in our area most often is space and the number of qualified judges. Take a competition two years ago in a neighboring state. I judged a 21 beer flight, followed by a14 beer flight, helped with 5 beers in a 19 beer flight and then did BOS. Why you ask? Lack of judges and a larger than expected turnout of entries. The particular state in question has a total of 5, yes, 5 BJCP judges in that state. North Carolina comps can have a difficult time finding judges and we have 55 in the state according to the demographics. While 8-10 beers is ideal, it is not particularly realistic at times although a 21 beer flight is far too much. As far as judging time. I think 10-12 minutes per beer is a good pace. If you judge slower than that, why? Are you talking too much, are you talking too little? I write the entire time while judging and try to fill in the aroma, flavor and overall impression without any remaining white space when possible. I see many judges socializing more than judging, and that is fine. No matter what, judging should be fun and rewarding and not simply making comments on a piece of paper. However you can have fun and still judge the beer in a timely fashion. For why people will not go below 19...I try to stay above 16 and actually as high as I can since those beers are more than likely a new brewer. We do not want them to stop brewing as a result of a bad judging sheet. I believe I have given a few 14 scores for badly infected beers. That still doesn't get them to the Problematic range of the scoresheet. Maybe using the entire range is something that should be explored. How about a troubleshooting guide of a single page that could be provided at comps to any beer that scored below a 20? As the Exam Study Guide is revised, perhaps those folks could dream up something like this. I'd be willing to assist as needed. Lastly, John mentioned his beer got a 37, someone else dubbed it a "mediocre score". On the scoresheets I use a 37 is the top of Very Good and if it scored a 38 it would be in the excellent range. A mediocre beer would be in the good range most likely 21-29. I remember my first beer that scored a 37 at competition. It was anything but mediocre. Came in 4th in the flight and bested every other beer of that style in the flight. It was extract and it took me quite a bit of experimenting to best with an AG batch. Cheers, Mike Dixon Wake Forest, NC BJCP National ---------------------------------------------------------------------- From: Chris Love Date: Wed, 24 May 2006 07:56:49 -0700 Subject: Call for judges - B3 2nd Annual Forum HBC Greetings: Mark your calendars for Saturday, June 10, 2006 as The 2nd Annual B3 Forum Homebrew Competition is now calling for judges and stewards. Judging will be at the beautiful E.J. Phair Brewing Company and Alehouse in Concord, California. A light breakfast and lunch will be provided. Event details: Saturday, June 10th - 9:30 AM to 3:00 PM. Judges and Stewards should arrive by 9:00 AM so judging can start promptly at 9:30. E.J. Phair Brewing Company and Alehouse http://www.ejphair.com/alehouse.htm 2151 Salvio Street Suite L Concord, CA 94520 If you are interested in judging or stewarding for this event please go to http://www.brewingcompetition.com/2006B3/2006B3.html and register now! Thank you and cheers! Chris Love Judge Coordinator chris`at`brew-monkey.com ---------------------------------------------------------------------- From: Dr. L.A. Swihart Date: Wed, 24 May 2006 14:54:28 -0400 Subject: 90/80/70/60.... Hi folks -- I am really appreciating this "accountability for 50 pints" discussion (OK, I mean points, whatever...;-), thanks to all for some very well considered and stated points. My impression since I started judging a couple years ago is that it's VERY hard for most people (both judges and competition entrants) to drag their expectations away from the old 90% = "the A cut-off" and 80% = the B cut-off, etc...... A push toward better quantitation is a fine idea, and it has my vote, although any effort to truly monitor it will not fly without hundreds of additional volunteer hours put into reviewing judge scoresheets and "grading" them, recording the grades, and considering and acting on the results. But before we could even attempt that, a prerequisite pressing issue is to make sure there is global and consistent buy-in for getting rid of the old 90% = A concept, and embrace, with our hearts and minds and pencils, the fact that the cut-offs are: 90% outstanding... world class 76% excellent... minor fine tuning 60% very good... some minor flaws 42% good ... 28% fair and score for each characteristic accordingly. Meaning that a beer could be deemed to have "very good" flavor and still receive a 12/20. $0.02, Linda ---------------------------------------------------------------------- From: Jon Tobey Date: Wed, 24 May 2006 11:58:47 -0700 (PDT) Subject: RE: Judging (John Peed et all) "Jon Tobey, clearly we are trying to pick the best beer AND help people make better beer. I don't speak in any official capacity, but it's clear to me that the goal is to do both. I think judging is too subjective now, and I believe that a lot of the subjectivity can be described objectively. I believe that accounting for your point deductions makes you do that. "Why am I giving this beer a 7 for aroma? Well, because I think it's OK - not great, but not bad. Why am I deducting those 5 points? Well, the malt aroma is a bit weak (-1). The hop aroma is really pretty good, but somewhat less than sensational .." First, based on the directions we give judges, and the feedback they give brewers, this has not been clear, so I'm glad you and others have stepped up and said this and I think we need to make this very clear at competitions. (This is what I hoped when I posed the question) Second, I wholeheartedly agree that the only way to do this is to account for point deductions, as you mention above. Why we don't mandate this is bewildering. Almost everytime I ask a co-judge about this in competition, the scores magically come up. I remember clearly a conversation with one individual "Well, because I always give a 2/3 for appearance..." !?) Inuititve, bottom-up judging is simply NOT objective, yet it is promoted as an acceptable way to do it. I wonder if that's how they score figure skating in the Olympics. "I just feels like 5.6 to me." Or term papers (You feel like a C student to me.) Or elections (This state feels red, doesn't it just feel red?)... We even have a built-in fudge category on the score sheet, fully 20% to rate the beer's overall impression that is not tied to the style guidelines. Clearly not designed to give an objective score. Lori's point on clipping the scale puts another fascinating twist on the whole thing that I'd never really put into words, and certainly talks to the bell curve approach (which, isn't really applicable here, see below) . I find it painfully dishonest to try to come up with a fabricated minimum score for a beer so poor it has no redeeming qualities. If competitor ego is the issue here, I'm sure giving a beer a 0 (or even just a "No Score" category), with appropriate constructive feedback would do just as well or better than a minimum score with insufficient feedback. Usually with a beer like that, it's pretty easy to tell them the one thing they need to do to make extreme improvement. That is, it doesn't take all day to give the feedback most useful. In other words, I think the instructions we are giving our judges make it difficult, if not impossible to give the most accurate, objective feedback. They seem tweaked to produce a mean artificial score that is really irrelevant to the stated purposes of making better beer - we don't care how a beer scores compared to other beers, we care how well it compares to an ideal. As an after effect we then compare scores to determine winners, but that has nothing to do with scoring each beer individually, so the proposed bell curve scoring idea is really a red herring. Think Olympics, not math test: We don't weight it so 2/3 get silver while the rest get bronze and gold. Well we do, but we shouldn't. We've done pretty well for a long time, but with our combined experience, we could be doing better. Jon Tobey Ideastream 425-822-8351 "It's like one of those craziass Australian wooden Frisbees." My Name is Earl ---------------------------------------------------------------------- From: Leo Koster Date: Wed, 24 May 2006 21:37:06 -0500 Subject: Re: accounting for points If you look at the score explanation at the bottom of the score sheet, you will find a brief, but fairly succinct description. If you are having a hard time with percentages being "fair", think of it as if you get a 50% for trying and the score you get is added to the 50%. Then your score of 40 becomes 90% - the bottom of the "A" range. Leo Koster ---------------------------------------------------------------------- ***** Important Subscriber Information ***** To post a message to JudgeNet, send it to judge`at`synchro.com. Send plain text only, no HTML, MIME, encoded text or attachments. Make sure you use a meaningful subject. Quote only as much material as is needed for context. To manage your subscription, go to http://synchro.com/judge/subscriptions.html or send an email to judge-request`at`synchro.com with the subject: help judge. JudgeNet is also available as an NNTP newsgroup, go to news://news.synchro.com/synchro.judge