From synchro!judge-request at uu6.psi.com Fri Jun 10 07:17:22 1994 Received: from uu6.psi.com by goodman.itn.med.umich.edu with SMTP id AA19416 (5.65b/IDA-1.4.3 for spencer at hendrix.itn.med.umich.edu); Fri, 10 Jun 94 07:17:12 -0400 Received: from synchro.UUCP by uu6.psi.com (5.65b/4.0.071791-PSI/PSINet) via UUCP; id AA24634 for ; Fri, 10 Jun 94 06:47:57 -0400 Received: by synchro.com (smail2.5) id AA11806; 10 Jun 94 05:30:32 EDT (Fri) Reply-To: judge at synchro.com (JudgeNet) Errors-To: judge-error at synchro.com Precedence: bulk Message-Id: <9406100530.AA11806 at synchro.com> From: judge-request at synchro.com (JudgeNet Administrator) To: judge-recipients at synchro.com (JudgeNet Recipients) Subject: JudgeNet Digest #782 (Jun 10, 1994) Date: 10 Jun 94 05:30:32 EDT (Fri) JudgeNet Digest #782 Fri 10 Jun 1994 THE BEER JUDGE DIGEST Chuck Cox , publisher Michael Hall , archive administrator digest submissions to judge at synchro.com administrative requests to judge-request at synchro.com send rank updates to the administrative address messages sent to the wrong address will be ignored FTP archive information in /pub/judge/README on cygnus.ta52.lanl.gov Published by SynchroSystems and the Riverside Garage & Brewery Contents: Re: quality of judges (John DeCarlo x7116 ) Malt flavor profiles (Spencer.W.Thomas) quality of judges (Spencer.W.Thomas) Complaints ("Rad Equipment") Complaints Time:7:44 AM Date:6/9/94 Judging (Ed Hitchcock) Re: judge critiques (Bill Slack) Judginr g quality (WESTEMEIER) ---------------------------------------------------------------------- Date: Thu, 9 Jun 94 08:24:28 EST From: John DeCarlo x7116 Subject: Re: quality of judges Michael Fetzer writes: >I have some suggestions to offer as well, and would like to hear from >you folks on that topic. >1. write sub point totals next to each sub category listed. E.g., if >you thought the hop aroma in that Am. Pale Ale was mediocre, write 1 >or 2 next to hops (out of 3). Yow! I can't see how this is feasible or even necessarily desirable. I guess you would have to agree on what the breakdown should be for each style (e.g., for Weizens, the aroma should have no points for hops and none for malt, all the aroma points should be for the esters and phenols). Constructive information makes sense (hop nose overpowers malt aroma which is perfect/unacceptable for style), but assigning that a point sounds weird. What points do you assign to off aromas like sourness or extra fruitiness (assuming they are not right for the style)? Negative points? All in all this sounds too complicated. >2. write a comment on everything, with the possible exception of >appearance (self explanatory if sub points are indicated). Good idea. I know I try but probably fail most of the time. >3. use constructive criticism. I know my beer is brown, cloudy, malty, >watery. Questions are: is it too dark for style? is it too cloudy for >style? should there be less malt for style? Should I make it stronger, >maltier, hoppier, add portland cement to give it body? I have not been very good in the "how to improve" section, so I am trying to improve. Mostly have been concentrating on "color is light copper, but should be much darker for a stout" type comments that give you a characteristic and whether that is right for the style. I will still hedge on advice, though. "Lacks malty flavor for style, maybe change your malt profile (more caramel malts) or possibly yeast or possibly mash schedule or ..." Since we don't know what the brewer did, it is hard to recommend a change. So when I advocate a change, I will probably only list one thing and hedge. After all, maybe they used all caramel malts but did something else to remove any malty sweetness. >4. somewhere on that form, be encouraging and supportive. Very important, IMHO. After all, part of being a BJCP judge is to encourage homebrewers with useful and positive feedback. John DeCarlo, MITRE Corporation, McLean, VA--My views are my own Fidonet: 1:109/131 Internet: jdecarlo at mitre.org ------------------------------ Date: Thu, 9 Jun 94 10:55:10 EDT From: Spencer.W.Thomas at med.umich.edu Subject: Malt flavor profiles [I sent this to the digest on May 31, but it never appeared. I'm trying again.] The local HB club will be doing a malt tasting at our next meeting. One of our members got the NW Malting "sample kit", which includes little bags of some 40 malts. I don't think we'll be going through all of them, though. :-) I'm thinking about putting together a "flavor profile" sheet so we can try to capture some "real data". It would ask for a rating (0-10, 1-5, or some such scale) for each of a number of flavor (aroma?) components, such as sweet, toasted, roasted, bitter, burnt, etc. Of course, many of these terms are quite subjective (i.e., toasted vs roasted), but I have hopes that some useful consensus might result. My main question at this point is: what flavor descriptors should I use? There will also be a blank for "comments" (e.g. to record "raisins!" for Special B), so the list should not be exhaustive. But I would like to cover the flavors commonly found in malts. Methodology will be very simple: everybody will crunch a small amount of the malt in their mouth, then record their impressions. =S ------------------------------ Date: Thu, 9 Jun 94 12:05:17 EDT From: Spencer.W.Thomas at med.umich.edu Subject: quality of judges I have to agree, that's pretty dismal judging performance. Some comments, in no particular order: First, you should complain to the BJCP administrator. The newer judging forms have this at the bottom: "How's my judging? ..." and tells you how to send complaints/compliments to the BJCP. In every competition I've judged, the judge coordinator has always emphasized that scores should be within 7, and that they should be adjusted if not. Don't know why that didn't happen here. Is there any correlation between the lack of comments/dumb comments and the level of the judge in BJCP? For example, if Beer 1/Judge 2 was an apprentice judge, his (her?) "copying" from Judge 1 would not be surprising. On "watery": I've tasted beers that, although full-bodied, still had a "watery" flavor, usually in the "middle". That is, the initial impression as the beer enters my mouth is good, and the aftertaste is good, but there's something missing in between. The only hypothesis I've formed, but not tested, is that perhaps a touch of salt would "fill it in". On breaking down scores: I tend not to do this, except in the appearance category, because when I do, I tend to overscore. I'm more of a top-down scorer, rather than bottom-up. If the comments are good, the score breakdown isn't as relevant, anyway. On comments: I'm with you on this one. Good comments are the most important part of the score sheet. I try to write down my impressions as they form (because flavor & aroma are such fleeting things), then to add some "analysis" if called for. Usually I do the "brewing improvement suggestions" in the overall impression section, rather than scattering them in the other sections, because most faults manifest in several areas (most commonly in aroma & flavor, both). I wish the OI section was about twice as big, because I frequently go onto the back of the sheet. I have to reluctantly agree (from my experience, too) that judges tend to like strong beers. I am actively fighting that tendency in my own judging, and encourage fellow judges in my flights to do likewise. I had two experiences, on both sides of this "coin", at the recent AHA first round in Chicago. On the good side, I had a great time judging Scottish Ales (Light, Heavy, Export). These range in gravity from 1.030 (Light) through 1.050 (Export). One Light made it into the top three from our flight, although it didn't pass the consolidation round with the other table. But at least one Heavy did (1.035-1.040 gravity, roughly). And not because we were trying to bias the results, but because it was a very well-made beer, *for its style*. On the down side, I judged stouts (1.038 - 1.090). There were 6 flights, with 2 going from each flight to the consolidation round. ALL of the lighter-bodied Dry Stouts were eliminated fairly quickly by the senior judges at the table as being "weak", "lacking flavor", etc. Now, granted, these were not the best examples of stout I've ever had (unfortunately), but neither were the Imperial Stouts. Of the three that went on to the 2nd round, two were Imperial Stouts and one was an Foreign style. To me, that's a clear bias. Yes, these were *BIG* flavorful beers, but I don't think they were really better Imperial stouts than some of the dry and sweet stouts were in their categories. But, as a relatively new judge, I found it hard to fight judges who have been at it for longer than I've been brewing. There is a lot of bad judging out there. I recently saw a collection of score sheets returned from a local competition (not, thank goodness, one in which I was involved). A friend's Belgian triple was given a score of 15, with the comments: "phenolic, probably infected". Right! I've tasted this beer. It's a pretty good example of a triple. A friend who was juding at another table says that this particular judge was commenting "I hate judging Belgians!" Well, he should have asked to be moved to a different category. Another judge gave a mead a 3, based on a single comment: "smells infected", with all the categories but appearance given a 0 score. It's clear he never even tasted it! ------------------------------ Date: 9 Jun 1994 08:58:01 -0800 From: "Rad Equipment" Subject: Complaints Subject: Complaints Time:7:44 AM Date:6/9/94 Re Michael Fetzer's comments on his competition feedback: Anyone who has a complaint about a competition or the judging quality thereof should contact the organizer, first and foremost! It is fine to cite examples of poor judging in forums like this one but you will be much more effective by sending these comments to the people who ran the competition in question. Supply copies of score sheets and indicate where you are dissatisfied. If your complaint is about a specific judge and he/she is BJCP affiliated you can send a complaint to the BJCP Co-directors via the BJCP Administrator at the AHA office in Boulder. Getting feedback like this is one of the few ways organizers get to evaluate the judges in their local pool. With competitions growing in size and the judging rolls expanding it is much harder for an organizer to know the abilities of each and every judge who responds to a mailing. And while it is suggested that organizers review score sheets for such information after the event, it is often in conflict with getting the results returned to the entrants in a timely manor. There is precious little feedback supplied to judges concerning how well they communicate with the entrants. This is a major flaw in the BJCP, IMHO. The rank earned by a BJCP judge only reflects how well the exam scoresheets were completed. There is no follow up to see how the judge performs in the real world. I know several very experienced judges (who's opinions I respect) who have turned in dismal scoresheets at competitions I have run. They get lazy, or bored, or whatever and forget that someone is going to take their comments (or lack of) very much to heart. Being called on the carpet is a very effective means of reminding them. In the absence of any ongoing evaluation of judges' performance we need to take advantage of the system as it stands and complain loudly (and to the right people) when we are dissatisfied. Now, having said all that, I do not think it is reasonable to expect a very detailed analysis of each entry. The notion that a judge can accurately troubleshoot a beer simply by tasting it once is ridiculous. The judge does not have any access to information as to recipe or process of a specific entry. Certainly judges can point out deviations with regard to style and identify problems with off flavors and such. Suggestions as to modifying ingredients or procedures should be offered but they are a shot in the dark at best. Praise should always be included if apropos. Also, there are generally 12 to 15 entries which need to be evaluated in a 2 hour period so the amount of time a judge can spend on each is limited. What you should expect from a judge is an unbiased opinion about your efforts. That includes thoughts independent of the other judges on the panel. Clear, intelligent comments to this end should be provided. RW... Russ Wigglesworth (INTERNET: Rad_Equipment at radmac1.ucsf.edu - CI$: 72300,61) UCSF Dept. of Radiology, San Francisco, CA (415) 476-3668 / Home (707) 769-0425 ------------------------------ Date: Thu, 09 Jun 1994 14:36:12 -0300 From: Ed Hitchcock Subject: Judging Michael Fetzer makes several comments about judging, to which I would like to reply. I agree that a judge should try to make positive comments, and I try to do so in at least the flavour and overall drinkability sections. If possible, I like to start with something like "Good malty character, but...", and proceed to highlight problem areas or noteworthy characteristics or whatever. However, after judging several beers it becomes increasingly difficult to nail anything specific in the aroma except for faults, so the aroma comments may start to get vague. I disagree that in lieu of comments a simple scoring breakdown is acceptable. There has been much discussion in this digest about allocating points, and I think you will find general agreement that you cannot make a one to one correlation with the scoring breakdown guidelines and the final score. Take, for example, a mediocre beer, one that would score about a 26. Looks great, body is fine, nothing off in the aroma, give it ~17 for those alone, because those are tangible, arguable points they must be delivered. But the beer is only about 26 points worth. So how do you break down the flavour and drinkability to give it 9 points? Well, I'm afraid you have to fudge it a little. Yes folks, fudge it. If every beer got all the points it deserved for every little subdivision of each catagory there would be a lot more 35-45's out there. As a quick anecdotal account in defence of top-down scoring, I was judging recently and in my third session of the day (Belgian, after Continental Pilsner and Stout) one beer was accidentally slipped in twice, once near the beginning, and once near the end. Guess what? I gave it a 25 both times. Oh, the individual scores were jsut a little different (a point here and there for aroma and overal drinkability and so on), but the comments were basically the same, and more importantly after 40 or so samples I could still say "This is about a 25" and work up the score from there. Some of Michael's sheets had no comments, which is just plain sloppy. I must admit thought, that the senses do become slightly muddled at some points during a session (usually just before the truly creative comments start flowing...), and it is sometimes hard to make comments. As for high point spreads, if the beer goes forward to a second round, the judges do not have the previous round's score sheets, so they may well judge more harshly or leniently, particularly if it gets up to the best of show. ____________ Ed Hitchcock ech at ac.dal.ca | Oxymoron: Draft beer in bottles. | Anatomy & Neurobiology | Pleonasm: Draft beer on tap. | Dalhousie University, Halifax |___________________________________| ------------------------------ Date: Thu, 9 Jun 94 17:36:19 EDT From: wslack!wrs (Bill Slack) Subject: Re: judge critiques I want to thank Scott Bickham not only for having nice things to say about how his beer was judged but also for providing balance to this forum by commenting on good judging practices when they show up. If you only hear about the complaints it gets discouraging after awhile. Not that there aren't real horror stories, as the other post in that digest showed. But it's just as useful to report good judging as it is to expose bad practices, and we should do more of it. IMHO, of course. Bill __ wrs at gozer.mv.com (Bill Slack) ------------------------------ Date: Thu, 09 Jun 1994 20:46:00 -0400 (EDT) From: WESTEMEIER at delphi.com Subject: Judginr g quality This is in reply to Michael Fetzer's thoughtful complaints about the level of comments he got on his beers. As the judge coordinator for the Cincinnati regionals, I am forced to concur with him in general, but plead guilty with an explanation. First, there was (at least at this regional site, a sever4e shortage of qualified judges. I tried hard to put at least one experienced judge on every flight, but it was not always possible. Leaving aside the issue of how well qualified a person is to judge a particular style, it was simply impossible to even assign a judge with experience to every flight. Sad, but true. We gfeel that there are simply not enough judges in this geographical area to warrant doing it agian next year. Second, we really did give instructions to the judges. Things like "Make comments in every section of the score sheet." "Balance a negative comment with a positive one." and "Make your comments helpful, not merely descriptive." Most judges did very well, but some were either too intimidated by judging for the first time (this group just went along with whatever the senior judge at the table said) or too self-centered to care how helpful their comments were. Overall, I would say that we had about 75% good score sheets, 20% poor ones, and 5% outstanding ones. IMHO, that pretty well covers the qualifications of BJCP judges on a national basis, so it's probably representative. As I see it, there is only one way to improve the situation. We need lots more judges in the BJCP, so that they can build up expereiience and get better. The only way for that to happen is to support the program basically as it is, and keep encouraging people to take the test and judge in competitions. Speaking for myself, I have learned more from being paired with more knowledgeable judges at competitions than I have in almost any other forum. It's a GOOD program. It's just too small (as yet) to be as good as it can be. I have my own horrible example to share, since I have the floor. I sent the exact same beer to two competitions recently. At one, a National judge (who has an extremely low reputation in the eyes of our local club members) scored it at 19, with the novice who assisted him concurring with a 20. The same beer, judgesd a few days later in another city, scored in the mid-30s consistently across 4 judges, who had nothing but good things to say about it. Face it, you're often at the mercy of circumstances about which you have no control. It's a lot loike being in front of a judge in a court of law. On any given day, anything can happen. Thanks for bringing up the topic, Mike. We know how you feel, and the best I can do at this point is to encourage you to have patience and do your best on a personal basis by judging as often as you can, and helping all of us to improve. Ed Westemeier Cincinnati, Ohio westemeier at delphi.com ------------------------------ End of JudgeNet Digest ************************