Return-Path: judge-owner at synchro.com Received: from srvr22.engin.umich.edu (root at srvr22.engin.umich.edu [141.212.2.35]) by srvr5.engin.umich.edu (8.8.8/8.8.8) with ESMTP id XAA21135 for ; Fri, 28 Aug 1998 23:05:03 -0400 (EDT) Received: from synchro.com (cccox.ne.mediaone.net [24.128.144.90]) by srvr22.engin.umich.edu (8.8.8/8.8.8) with SMTP id XAA28856 for ; Fri, 28 Aug 1998 23:05:02 -0400 (EDT) From: "JudgeNet - the beer judge digest" To: "Digest Recipients" Reply-To: "JudgeNet - the beer judge digest" Subject: Digest for the period 8/28/98 - 8/28/98 Message-ID: Date: Fri, 28 Aug 1998 23:02:12 -0400 MIME-Version: 1.0 Content-Type: multipart/digest; boundary="Next_Part_SYNC48631C28C" Content-Transfer-Encoding: 7bit X-Hops: 1 --Next_Part_SYNC48631C28C Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Table of contents ------------------------------------------------------ apologies (Chuck Cox) Question - Selecting Beers for BOS (Greg Lorton) --Next_Part_SYNC48631C28C Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48551BDAE; Fri, 28 Aug 1998 15:56:19 -0400 Received: from smtp0-alterdial.uu.net (smtp0-alterdial.UU.NET [192.48.96.28]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id PAA05658 for ; Fri, 28 Aug 1998 15:55:13 -0400 (EDT) Received: from conjunto by smtp0-alterdial.uu.net with SMTP (peer crosschecked as: 1Cust240.tnt3.bos1.da.uu.net [153.34.162.240]) id QQfekt14081; Fri, 28 Aug 1998 19:55:31 GMT Message-Id: <3.0.3.32.19980828154350.0068c0fc at mail59367.popserver.pop.net> X-Sender: mail59367 at mail59367.popserver.pop.net X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.3 (32) Date: Fri, 28 Aug 1998 15:43:50 -0400 To: JudgeNet - the beer judge digest Sender: judge at synchro.com From: Chuck Cox Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Subject: apologies Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-UIDL: 090433412605662 X-Hops: 1 X-POST-MessageClass: 10; Mailing List Message Sorry about that non-judge posting in the last digest. That message managed to slip past the moderation mechanism. I have taken steps to prevent a recurrance. Feel free to express your opinion about spam to the originator. - Chuck Cox - SynchroSystems --Next_Part_SYNC48631C28C Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48551BD3D; Fri, 28 Aug 1998 15:16:51 -0400 Received: from mh2.cts.com (mh2.cts.com [209.68.192.68]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id PAA27904 for ; Fri, 28 Aug 1998 15:13:29 -0400 (EDT) Received: from pavilion (psc13221011.cts.com [204.216.221.11]) by mh2.cts.com (8.8.7/8.8.5) with ESMTP id MAA25373 for ; Fri, 28 Aug 1998 12:13:34 -0700 (PDT) Message-Id: <199808281913.MAA25373 at mh2.cts.com> Reply-To: "JudgeNet - the beer judge digest" From: "Greg Lorton" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com To: JudgeNet - the beer judge digest Subject: Question - Selecting Beers for BOS Date: Fri, 28 Aug 1998 12:06:45 -0700 X-MSMail-Priority: Normal X-Priority: 3 X-Mailer: Microsoft Internet Mail 4.70.1162 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-UIDL: 090433161827908 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Since JudgeNet has been a little slow lately, and has recently taken to advertising for non-mainstream political viewpoints, I thought I'd suggest a discussion related to homebrew competitions. I'm curious how large competitions deal with selecting beers for the Best of Show round. I'd like some viewpoints and suggestions on how to deal with this. Background: The America's Finest City Homebrew Competition in San Diego is a large contest (250 to 300 entries). We need 40 to 50 judges. The majority of the judges come from out-of-town (especially the LA area), so we try to do as much of the judging as we can on one day (Saturday), since most judges want to come and go on the same day. We usually have entries in all categories, and can potentially have 28 first-place beers qualify as candidates for Best of Show. The BJCP competition guidelines allow the organizer to give BOS judging points (1/2 points) to 5 judges. We prefer to have out-of-town judges on the BOS panel, in appreciation of their effort to help us. Since BOS judging takes place in the mid to late afternoon, there is quite a bit of incentive to cull the number of candidates so that everything can get wrapped up. What we've done in the past is to take only the beers with the highest scores, and allow them to move to the BOS round. (This is in addition to our rule that a beer must get a score of 35 to win a first-place award.) For the last contest, we took the top 16 scoring first place beers (scores of 38 or higher). Since we had 23 first-place beers (minimum score of 35), that meant that 7 beers were not considered for BOS. (and yours truly owned one of those 7) :-) Doing this keeps the number of BOS candidates to a semi-manageable number, but it does penalize those first place beers that didn't score high enough. Maybe those beers weren't really BOS caliber, but on the other hand, maybe the judges on that panel were very tough graders. What strategies are used by other large competitions in selecting BOS candidates? Are all first-place beers advanced to the BOS round? If so, how are the BOS judging chores carried out? Are high scoring second- or third-place beers ever considered for BOS (kind of like second-and third-place first round beers in AHA NHC ultimately winning their category in the second round)? Cheers! Greg Lorton Carlsbad, CA glorton at cts.com --Next_Part_SYNC48631C28C-- Return-Path: judge-owner at synchro.com Received: from srvr22.engin.umich.edu (root at srvr22.engin.umich.edu [141.212.2.35]) by srvr5.engin.umich.edu (8.8.8/8.8.8) with ESMTP id XAA21135 for ; Fri, 28 Aug 1998 23:05:03 -0400 (EDT) Received: from synchro.com (cccox.ne.mediaone.net [24.128.144.90]) by srvr22.engin.umich.edu (8.8.8/8.8.8) with SMTP id XAA28856 for ; Fri, 28 Aug 1998 23:05:02 -0400 (EDT) From: "JudgeNet - the beer judge digest" To: "Digest Recipients" Reply-To: "JudgeNet - the beer judge digest" Subject: Digest for the period 8/28/98 - 8/28/98 Message-ID: Date: Fri, 28 Aug 1998 23:02:12 -0400 MIME-Version: 1.0 Content-Type: multipart/digest; boundary="Next_Part_SYNC48631C28C" Content-Transfer-Encoding: 7bit X-Hops: 1 --Next_Part_SYNC48631C28C Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Table of contents ------------------------------------------------------ apologies (Chuck Cox) Question - Selecting Beers for BOS (Greg Lorton) --Next_Part_SYNC48631C28C Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48551BDAE; Fri, 28 Aug 1998 15:56:19 -0400 Received: from smtp0-alterdial.uu.net (smtp0-alterdial.UU.NET [192.48.96.28]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id PAA05658 for ; Fri, 28 Aug 1998 15:55:13 -0400 (EDT) Received: from conjunto by smtp0-alterdial.uu.net with SMTP (peer crosschecked as: 1Cust240.tnt3.bos1.da.uu.net [153.34.162.240]) id QQfekt14081; Fri, 28 Aug 1998 19:55:31 GMT Message-Id: <3.0.3.32.19980828154350.0068c0fc at mail59367.popserver.pop.net> X-Sender: mail59367 at mail59367.popserver.pop.net X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.3 (32) Date: Fri, 28 Aug 1998 15:43:50 -0400 To: JudgeNet - the beer judge digest Sender: judge at synchro.com From: Chuck Cox Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Subject: apologies Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-UIDL: 090433412605662 X-Hops: 1 X-POST-MessageClass: 10; Mailing List Message Sorry about that non-judge posting in the last digest. That message managed to slip past the moderation mechanism. I have taken steps to prevent a recurrance. Feel free to express your opinion about spam to the originator. - Chuck Cox - SynchroSystems --Next_Part_SYNC48631C28C Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48551BD3D; Fri, 28 Aug 1998 15:16:51 -0400 Received: from mh2.cts.com (mh2.cts.com [209.68.192.68]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id PAA27904 for ; Fri, 28 Aug 1998 15:13:29 -0400 (EDT) Received: from pavilion (psc13221011.cts.com [204.216.221.11]) by mh2.cts.com (8.8.7/8.8.5) with ESMTP id MAA25373 for ; Fri, 28 Aug 1998 12:13:34 -0700 (PDT) Message-Id: <199808281913.MAA25373 at mh2.cts.com> Reply-To: "JudgeNet - the beer judge digest" From: "Greg Lorton" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com To: JudgeNet - the beer judge digest Subject: Question - Selecting Beers for BOS Date: Fri, 28 Aug 1998 12:06:45 -0700 X-MSMail-Priority: Normal X-Priority: 3 X-Mailer: Microsoft Internet Mail 4.70.1162 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-UIDL: 090433161827908 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Since JudgeNet has been a little slow lately, and has recently taken to advertising for non-mainstream political viewpoints, I thought I'd suggest a discussion related to homebrew competitions. I'm curious how large competitions deal with selecting beers for the Best of Show round. I'd like some viewpoints and suggestions on how to deal with this. Background: The America's Finest City Homebrew Competition in San Diego is a large contest (250 to 300 entries). We need 40 to 50 judges. The majority of the judges come from out-of-town (especially the LA area), so we try to do as much of the judging as we can on one day (Saturday), since most judges want to come and go on the same day. We usually have entries in all categories, and can potentially have 28 first-place beers qualify as candidates for Best of Show. The BJCP competition guidelines allow the organizer to give BOS judging points (1/2 points) to 5 judges. We prefer to have out-of-town judges on the BOS panel, in appreciation of their effort to help us. Since BOS judging takes place in the mid to late afternoon, there is quite a bit of incentive to cull the number of candidates so that everything can get wrapped up. What we've done in the past is to take only the beers with the highest scores, and allow them to move to the BOS round. (This is in addition to our rule that a beer must get a score of 35 to win a first-place award.) For the last contest, we took the top 16 scoring first place beers (scores of 38 or higher). Since we had 23 first-place beers (minimum score of 35), that meant that 7 beers were not considered for BOS. (and yours truly owned one of those 7) :-) Doing this keeps the number of BOS candidates to a semi-manageable number, but it does penalize those first place beers that didn't score high enough. Maybe those beers weren't really BOS caliber, but on the other hand, maybe the judges on that panel were very tough graders. What strategies are used by other large competitions in selecting BOS candidates? Are all first-place beers advanced to the BOS round? If so, how are the BOS judging chores carried out? Are high scoring second- or third-place beers ever considered for BOS (kind of like second-and third-place first round beers in AHA NHC ultimately winning their category in the second round)? Cheers! Greg Lorton Carlsbad, CA glorton at cts.com --Next_Part_SYNC48631C28C-- Return-Path: judge-owner at synchro.com Received: from srvr20.engin.umich.edu (root at srvr20.engin.umich.edu [141.212.2.26]) by srvr5.engin.umich.edu (8.8.8/8.8.8) with ESMTP id XAA14593 for ; Sat, 29 Aug 1998 23:02:39 -0400 (EDT) Received: from synchro.com (cccox.ne.mediaone.net [24.128.144.90]) by srvr20.engin.umich.edu (8.8.8/8.8.8) with SMTP id XAA03153 for ; Sat, 29 Aug 1998 23:02:35 -0400 (EDT) From: "JudgeNet - the beer judge digest" To: "Digest Recipients" Reply-To: "JudgeNet - the beer judge digest" Subject: Digest for the period 8/28/98 - 8/29/98 Message-ID: Date: Sat, 29 Aug 1998 23:00:45 -0400 MIME-Version: 1.0 Content-Type: multipart/digest; boundary="Next_Part_SYNC48871D1DB" Content-Transfer-Encoding: 7bit X-Hops: 1 --Next_Part_SYNC48871D1DB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Table of contents ------------------------------------------------------ BOS Judging (Dan Sullivan) Selecting BOS (Jeremy Bergsman) BOS judging (PDWaltman) Question - Selecting Beers for BOS (David Houseman) Re: Selecting Beers for BOS (David Sherfey) --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48631C518; Fri, 28 Aug 1998 23:58:20 -0400 Received: from daffy.napanet.net (root at daffy.napanet.net [157.22.192.18]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id XAA13535 for ; Fri, 28 Aug 1998 23:54:19 -0400 (EDT) Received: from pm-n1-23.napanet.net (pm-n1-23.napanet.net [157.22.193.23]) by daffy.napanet.net (8.8.8/8.8.8) with SMTP id UAA19440 for ; Fri, 28 Aug 1998 20:54:38 -0700 X-Authentication-Warning: daffy.napanet.net: pm-n1-23.napanet.net [157.22.193.23] didn't use HELO protocol Message-ID: <35E77C05.443E at napanet.net> Date: Fri, 28 Aug 1998 20:56:53 -0700 From: Dan Sullivan Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Organization: Northern California Homebrewers Festival (NCHF) X-Mailer: Mozilla 3.01 (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: BOS Judging Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-UIDL: 090436289013540 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Greg allowed us to go into the weekend with this on our minds: "Are all first-place beers advanced to the BOS round? If so, how are the BOS judging chores carried out?" I'll pass on the 1st question, but on the second, I now have an all-time fave: The beers are brought out in fairly rapid, but orderly, succession, poured by the chief steward ably assisted by his BOS stewards. Judges are advised of the order of the beers and their styles before evaluation is begun, so that they may make necessary notations in preparation for judging. With a confirmatory announcement of the entry number and style, the beers are passed to the judges, who then capture the elusive aromatics (and perhaps a brief evaluatory sip) for quick note. Once the beers have been poured, the judges then go on to finish their evaluations, selecting their personal top 5 beers. Once that task is completed, they show their cards. Any one beer receiving votes from all 5 is a candidate for BOS. If only one beer garners top 5 votes, then that is likely the BOS, and should be either confirmed or argued. I have yet to see one receiving 5 votes not unanimously acclaimed BOS. Should there be no one beer with 5 votes, is there one with 4? Hey, YMMV, but it is a very agreeable method, which produces fairly uniform and unanimous results. Cal State Fair BOS was a recent example. One beer had 5 top 5 votes, 3 of which were #1. The second place beer had either 3 or 4 votes, 1 of which was #1. We all ended up agreeing that the winner was the best example of style from among the beers presented, and my best recollection was that there were 22 or 23. At least that's the way I kinda remember it.... It works. Cheers. Dan --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48641C542; Sat, 29 Aug 1998 00:22:53 -0400 Received: from mailhub.Stanford.EDU (mailhub.Stanford.EDU [171.64.14.35]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id AAA18198 for ; Sat, 29 Aug 1998 00:19:56 -0400 (EDT) Received: from leland.stanford.edu (tip-mp18-ncs-12.Stanford.EDU [36.173.1.123]) by mailhub.Stanford.EDU (8.8.8/8.8.8/L) with ESMTP id VAA06421 for ; Fri, 28 Aug 1998 21:20:13 -0700 (PDT) Message-ID: <35E78269.FFE3D6E1 at leland.stanford.edu> Date: Fri, 28 Aug 1998 21:24:09 -0700 From: Jeremy Bergsman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Organization: Stanford University X-Mailer: Mozilla 4.05 [en] (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: Selecting BOS References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-UIDL: 090436441218203 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message While not everyone will consider that a non-mainstream political viewpoint (spam does seem a bit odd from a libertarian) I'll address Greg's question of selecting beers for BOS. Basically he asks: 1) How can one minimize the number of BOS beers given a lot of categories? 2) Should all first place beers go to BOS or may one pick the highest N beers based on score? 3) Might one want to go the other way and include 2nd and 3rd place beers in BOS? I'll discuss (3) first. Since most people agree with the concern in (1) I see no reason to pass more than the first place beer from each category. You should trust your judges to pick #1. The reason you need to pass 3 beers from a prelim round to mini-BOS is that you want to be able to pick 1-3 overall and one of the several prelim flights might have all 3 best beers. Also, in some cases, prelim judging is done in a hurried fashion and the judges may not have carefully picked 1 2 and 3. None of this applies to BOS, assuming only #1 will be chosen (i.e. there is no second place in BOS). Next, (2). The strategy of picking BOS candidates by score is wrong. Period. Judges are not calibrated with each other well enough to do this. I actually have data to back up this statement. I participated in what was called, I believe, the "Beer Judge Calibration Project" which has been mentioned here a few times. While there were a few problems with the project, the major finding was, IMneverHO, that beer judges are all over the map with regard to their numerical scoring. For example, here are the data from the first 2 months: Beer Month Mean Max Min SD N 1 1 30.9 41 20 5.0 44 2 1 33.2 42 22 4.9 44 3 1 32.1 45 18 5.9 44 4 2 30.2 41 22 4.6 43 5 2 24.9 36 14 5.0 41 6 2 34.6 42 18 5.3 43 As you can see, any given beer can get a score that is very low or very high. Just as on normal judging panels, we became more internally calibrated as the months went on, but when comparing between flights for BOS there is no movement towards intraflight consistency. One problem with these data is that the beers could have been handled and/or served differently. I recently collected the following data for 2 beers served blind to 15 judges--3 novices (one very experienced) and 12 BJCP--in the same place at the same time with no discussion among judges: Beer Mean Max Min SD N 1 32.7 39* 22 7.1 15 2 31.4 43* 20 7.8 15 * One novice judge gave these beers 46 and 49, but since s/he is obviously more uncalibrated than most I have given the second-from-the-max here. Unfortunately I don't have advice for (1), but I can say I have been on two BOS rounds in the last year that dealt with a large number of beers without too much problem. I have seen that the following strategy works fairly well: judges taste all the beers while making brief notes, most likely about subtle problems. One can do more than 1 beer/minute this way. The judges then call out their top 2 or 3 beers. Usually this will narrow it down to 2 or 3 beers which can then receive more intense tasting and discussion. -- Jeremy Bergsman jeremybb at leland.stanford.edu http://www-leland.stanford.edu/~jeremybb --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48701C79E; Sat, 29 Aug 1998 06:33:31 -0400 Received: from dewdrop2.mindspring.com (dewdrop2.mindspring.com [207.69.200.82]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id GAA03456 for ; Sat, 29 Aug 1998 06:30:33 -0400 (EDT) Received: from default (user-37kbo52.dialup.mindspring.com [207.69.224.162]) by dewdrop2.mindspring.com (8.8.5/8.8.5) with SMTP id GAA22833 for ; Sat, 29 Aug 1998 06:30:54 -0400 (EDT) Message-Id: <3.0.1.32.19980829063232.006ac7c4 at pop.mindspring.com> X-Sender: awapuhoq at pop.mindspring.com X-Mailer: Windows Eudora Light Version 3.0.1 (32) Date: Sat, 29 Aug 1998 06:32:32 -0400 To: JudgeNet - the beer judge digest From: PDWaltman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Subject: BOS judging Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-UIDL: 090438663503459 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message On the subject of BOS judging, I'm sure the persons who actually did it and used it can describe it better, but I heard how BOS (or HBY) judging was handled quickly at the NHC 2nd round in Portland. There they had 28 beers and meads in 28 categories. they sampled the 28 beers writing quick notes on each without commenting to each other. Then they were each given some tokens (I think 3 of them). Then they were told to put a token on their choice for the top 3 beers/meads. Those beers with the most tokens were kept in the running. Now they just had to look for BOS or HBY, there was no 2nd place BOS or third place BOS, so I could imagine keeping the top X tokens, and going from there. Of course if one beer or mead got 5 tokens and the next highest got 3 tokens, you might have a clear idea of the winner right from the start. I'm thinking about doing something similar for the competition I'm organizing (a first timer, I knew taking the BJCP exam would cause me to do foolish things), as the BOS round always seems to come when the patrons of the establishment we are in wants us out. Dennis Waltman Peach State Brew Off - Atlanta, GA; November 7th, 1998 (any help very appreciated) --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48711C7DA; Sat, 29 Aug 1998 07:24:44 -0400 Received: from priam.chesco.net (priam.chesco.net [204.108.253.90]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id HAA06985 for ; Sat, 29 Aug 1998 07:22:51 -0400 (EDT) Received: from TR-dlhousemanREM.tr.unisys.com (shiva2.btcwcu.org [204.108.253.141]) by priam.chesco.net (8.8.8/8.8.8) with SMTP id HAA12860 for ; Sat, 29 Aug 1998 07:23:09 -0400 (EDT) Date: Sat, 29 Aug 1998 07:23:09 -0400 (EDT) Message-Id: <199808291123.HAA12860 at priam.chesco.net> X-Sender: dhousema at mail.cccbi.org X-Mailer: Windows Eudora Light Version 1.5.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: JudgeNet - the beer judge digest From: David Houseman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Subject: Question - Selecting Beers for BOS X-UIDL: 090438977206988 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message I'm sorry but I don't believe one can have an absolute numerical cutoff for qualifying for BOS as Greg suggests. The scoring within any given flight of judges can vary considerably; it's relative to the those judges. While the guidelines [at the bottom of the score sheet] do give a range of scores for GOOD, EXCELLENT, etc., these are in the opinion of the judges scoring. If we were measuring with something totally objective such as a metric ruler or volumetric flask, then, yes, you could have a cut off. However, judging by its nature is subjective. My flight's winner with a score of 32 may be superior to your flight's winner with a score of 39 even if it is the same style. Looking for BOS qualities is even more precarious to make the assumption that numerical values can be used as absolutes. The BUZZ OFF is also a fairly large competition with 501 entries this year. We sent all first place winners to the BOS round including cider and mead (we've had a cider win BOS but then it went on to win cidermaker of the year for the brewer so it really did deserve to win). Five judges judged the 27 or so categories without difficulty and time wasn't all that long although I didn't measure it. I've judged BOS at the Nationals where we had all beer categories. Juding all of them didn't take that much longer than if we'd started with the best 8 because the majority of the time was selecting the winner from the last three. Discarding the first 16 didn't take much time at all. Dave Houseman --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48731C857; Sat, 29 Aug 1998 09:13:17 -0400 Received: from host.warwick.net (host.warwick.net [204.255.24.254]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id JAA20147 for ; Sat, 29 Aug 1998 09:12:12 -0400 (EDT) Received: from warwick.net (w171-23.warwick.net [208.199.17.33]) by host.warwick.net (8.9.1/8.9.1) with ESMTP id IAA28760 for ; Sat, 29 Aug 1998 08:58:10 -0400 (EDT) Message-ID: <35E7FCFF.F255ADE9 at warwick.net> Date: Sat, 29 Aug 1998 09:07:12 -0400 From: David Sherfey Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Reply-To: "JudgeNet - the beer judge digest" X-Mailer: Mozilla 4.04 [en] (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: Re: Selecting Beers for BOS References: Content-Type: text/plain; charset=x-user-defined Content-Transfer-Encoding: 7bit X-UIDL: 090439633320150 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Greg-- If a competition informs it's entrants in advance of the BOS policy, it should be OK to handle BOS any way it sees fit. If its a bad policy, the entrants will let the organizers know it by not sending their entries to the competition. IMO, your policy of setting criteria for first place is excellent. I believe that most judges have the ability to determine if an entry is first place or not, and competitions should encourage them to make that call. Just because an entry is the "best" of a flight does not make it a first place beer. There have been occasions in the past where there were barely second place entries in categories I have judged and awarding any of them first place would have sent the wrong message to the brewer. The award is part of the feedback judges give the brewers, and it has to be honest. I believe your policy of limiting the quantity of entries getting to BOS by dropping off the lowest scores is going too far. The limiting criteria for first place should take care of weak entries getting to BOS. All first place entries should proceed to the BOS round. Entrants expect this and especially the individuals working the competition and have beers entered like to see their beers go as far as they can without editorial bias. BOS judges get those extra 1/2 points for that extra effort, and the number of entries they get is just part of the territory! However, since some competitions are held in venues that must be cleared for other activities, the time used at BOS judging needs to be minimized. One way to do this is to split the field in two and have two panels each judge half of the entries and then finish with a taste-off of the best six. Having only five judges makes this a little off center, with an extra judge on one panel. Perhaps the BJCP could allow a change to six judges? I doubt that many judges would pass up the opportunity to judge BOS just because there are 24 entries to judge and they have to drive back to LA or beyond. I judged BOS at one of your past competitions and we still had time to go out for dinner, go over to somone's house for a party, and drive home to LA that night (I drove). It's tough work and we just have to do it. Cheers! David Sherfey Warwick, NY --Next_Part_SYNC48871D1DB-- Return-Path: judge-owner at synchro.com Received: from srvr20.engin.umich.edu (root at srvr20.engin.umich.edu [141.212.2.26]) by srvr5.engin.umich.edu (8.8.8/8.8.8) with ESMTP id XAA14593 for ; Sat, 29 Aug 1998 23:02:39 -0400 (EDT) Received: from synchro.com (cccox.ne.mediaone.net [24.128.144.90]) by srvr20.engin.umich.edu (8.8.8/8.8.8) with SMTP id XAA03153 for ; Sat, 29 Aug 1998 23:02:35 -0400 (EDT) From: "JudgeNet - the beer judge digest" To: "Digest Recipients" Reply-To: "JudgeNet - the beer judge digest" Subject: Digest for the period 8/28/98 - 8/29/98 Message-ID: Date: Sat, 29 Aug 1998 23:00:45 -0400 MIME-Version: 1.0 Content-Type: multipart/digest; boundary="Next_Part_SYNC48871D1DB" Content-Transfer-Encoding: 7bit X-Hops: 1 --Next_Part_SYNC48871D1DB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Table of contents ------------------------------------------------------ BOS Judging (Dan Sullivan) Selecting BOS (Jeremy Bergsman) BOS judging (PDWaltman) Question - Selecting Beers for BOS (David Houseman) Re: Selecting Beers for BOS (David Sherfey) --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48631C518; Fri, 28 Aug 1998 23:58:20 -0400 Received: from daffy.napanet.net (root at daffy.napanet.net [157.22.192.18]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id XAA13535 for ; Fri, 28 Aug 1998 23:54:19 -0400 (EDT) Received: from pm-n1-23.napanet.net (pm-n1-23.napanet.net [157.22.193.23]) by daffy.napanet.net (8.8.8/8.8.8) with SMTP id UAA19440 for ; Fri, 28 Aug 1998 20:54:38 -0700 X-Authentication-Warning: daffy.napanet.net: pm-n1-23.napanet.net [157.22.193.23] didn't use HELO protocol Message-ID: <35E77C05.443E at napanet.net> Date: Fri, 28 Aug 1998 20:56:53 -0700 From: Dan Sullivan Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Organization: Northern California Homebrewers Festival (NCHF) X-Mailer: Mozilla 3.01 (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: BOS Judging Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-UIDL: 090436289013540 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Greg allowed us to go into the weekend with this on our minds: "Are all first-place beers advanced to the BOS round? If so, how are the BOS judging chores carried out?" I'll pass on the 1st question, but on the second, I now have an all-time fave: The beers are brought out in fairly rapid, but orderly, succession, poured by the chief steward ably assisted by his BOS stewards. Judges are advised of the order of the beers and their styles before evaluation is begun, so that they may make necessary notations in preparation for judging. With a confirmatory announcement of the entry number and style, the beers are passed to the judges, who then capture the elusive aromatics (and perhaps a brief evaluatory sip) for quick note. Once the beers have been poured, the judges then go on to finish their evaluations, selecting their personal top 5 beers. Once that task is completed, they show their cards. Any one beer receiving votes from all 5 is a candidate for BOS. If only one beer garners top 5 votes, then that is likely the BOS, and should be either confirmed or argued. I have yet to see one receiving 5 votes not unanimously acclaimed BOS. Should there be no one beer with 5 votes, is there one with 4? Hey, YMMV, but it is a very agreeable method, which produces fairly uniform and unanimous results. Cal State Fair BOS was a recent example. One beer had 5 top 5 votes, 3 of which were #1. The second place beer had either 3 or 4 votes, 1 of which was #1. We all ended up agreeing that the winner was the best example of style from among the beers presented, and my best recollection was that there were 22 or 23. At least that's the way I kinda remember it.... It works. Cheers. Dan --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48641C542; Sat, 29 Aug 1998 00:22:53 -0400 Received: from mailhub.Stanford.EDU (mailhub.Stanford.EDU [171.64.14.35]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id AAA18198 for ; Sat, 29 Aug 1998 00:19:56 -0400 (EDT) Received: from leland.stanford.edu (tip-mp18-ncs-12.Stanford.EDU [36.173.1.123]) by mailhub.Stanford.EDU (8.8.8/8.8.8/L) with ESMTP id VAA06421 for ; Fri, 28 Aug 1998 21:20:13 -0700 (PDT) Message-ID: <35E78269.FFE3D6E1 at leland.stanford.edu> Date: Fri, 28 Aug 1998 21:24:09 -0700 From: Jeremy Bergsman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Organization: Stanford University X-Mailer: Mozilla 4.05 [en] (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: Selecting BOS References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-UIDL: 090436441218203 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message While not everyone will consider that a non-mainstream political viewpoint (spam does seem a bit odd from a libertarian) I'll address Greg's question of selecting beers for BOS. Basically he asks: 1) How can one minimize the number of BOS beers given a lot of categories? 2) Should all first place beers go to BOS or may one pick the highest N beers based on score? 3) Might one want to go the other way and include 2nd and 3rd place beers in BOS? I'll discuss (3) first. Since most people agree with the concern in (1) I see no reason to pass more than the first place beer from each category. You should trust your judges to pick #1. The reason you need to pass 3 beers from a prelim round to mini-BOS is that you want to be able to pick 1-3 overall and one of the several prelim flights might have all 3 best beers. Also, in some cases, prelim judging is done in a hurried fashion and the judges may not have carefully picked 1 2 and 3. None of this applies to BOS, assuming only #1 will be chosen (i.e. there is no second place in BOS). Next, (2). The strategy of picking BOS candidates by score is wrong. Period. Judges are not calibrated with each other well enough to do this. I actually have data to back up this statement. I participated in what was called, I believe, the "Beer Judge Calibration Project" which has been mentioned here a few times. While there were a few problems with the project, the major finding was, IMneverHO, that beer judges are all over the map with regard to their numerical scoring. For example, here are the data from the first 2 months: Beer Month Mean Max Min SD N 1 1 30.9 41 20 5.0 44 2 1 33.2 42 22 4.9 44 3 1 32.1 45 18 5.9 44 4 2 30.2 41 22 4.6 43 5 2 24.9 36 14 5.0 41 6 2 34.6 42 18 5.3 43 As you can see, any given beer can get a score that is very low or very high. Just as on normal judging panels, we became more internally calibrated as the months went on, but when comparing between flights for BOS there is no movement towards intraflight consistency. One problem with these data is that the beers could have been handled and/or served differently. I recently collected the following data for 2 beers served blind to 15 judges--3 novices (one very experienced) and 12 BJCP--in the same place at the same time with no discussion among judges: Beer Mean Max Min SD N 1 32.7 39* 22 7.1 15 2 31.4 43* 20 7.8 15 * One novice judge gave these beers 46 and 49, but since s/he is obviously more uncalibrated than most I have given the second-from-the-max here. Unfortunately I don't have advice for (1), but I can say I have been on two BOS rounds in the last year that dealt with a large number of beers without too much problem. I have seen that the following strategy works fairly well: judges taste all the beers while making brief notes, most likely about subtle problems. One can do more than 1 beer/minute this way. The judges then call out their top 2 or 3 beers. Usually this will narrow it down to 2 or 3 beers which can then receive more intense tasting and discussion. -- Jeremy Bergsman jeremybb at leland.stanford.edu http://www-leland.stanford.edu/~jeremybb --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48701C79E; Sat, 29 Aug 1998 06:33:31 -0400 Received: from dewdrop2.mindspring.com (dewdrop2.mindspring.com [207.69.200.82]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id GAA03456 for ; Sat, 29 Aug 1998 06:30:33 -0400 (EDT) Received: from default (user-37kbo52.dialup.mindspring.com [207.69.224.162]) by dewdrop2.mindspring.com (8.8.5/8.8.5) with SMTP id GAA22833 for ; Sat, 29 Aug 1998 06:30:54 -0400 (EDT) Message-Id: <3.0.1.32.19980829063232.006ac7c4 at pop.mindspring.com> X-Sender: awapuhoq at pop.mindspring.com X-Mailer: Windows Eudora Light Version 3.0.1 (32) Date: Sat, 29 Aug 1998 06:32:32 -0400 To: JudgeNet - the beer judge digest From: PDWaltman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Subject: BOS judging Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-UIDL: 090438663503459 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message On the subject of BOS judging, I'm sure the persons who actually did it and used it can describe it better, but I heard how BOS (or HBY) judging was handled quickly at the NHC 2nd round in Portland. There they had 28 beers and meads in 28 categories. they sampled the 28 beers writing quick notes on each without commenting to each other. Then they were each given some tokens (I think 3 of them). Then they were told to put a token on their choice for the top 3 beers/meads. Those beers with the most tokens were kept in the running. Now they just had to look for BOS or HBY, there was no 2nd place BOS or third place BOS, so I could imagine keeping the top X tokens, and going from there. Of course if one beer or mead got 5 tokens and the next highest got 3 tokens, you might have a clear idea of the winner right from the start. I'm thinking about doing something similar for the competition I'm organizing (a first timer, I knew taking the BJCP exam would cause me to do foolish things), as the BOS round always seems to come when the patrons of the establishment we are in wants us out. Dennis Waltman Peach State Brew Off - Atlanta, GA; November 7th, 1998 (any help very appreciated) --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48711C7DA; Sat, 29 Aug 1998 07:24:44 -0400 Received: from priam.chesco.net (priam.chesco.net [204.108.253.90]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id HAA06985 for ; Sat, 29 Aug 1998 07:22:51 -0400 (EDT) Received: from TR-dlhousemanREM.tr.unisys.com (shiva2.btcwcu.org [204.108.253.141]) by priam.chesco.net (8.8.8/8.8.8) with SMTP id HAA12860 for ; Sat, 29 Aug 1998 07:23:09 -0400 (EDT) Date: Sat, 29 Aug 1998 07:23:09 -0400 (EDT) Message-Id: <199808291123.HAA12860 at priam.chesco.net> X-Sender: dhousema at mail.cccbi.org X-Mailer: Windows Eudora Light Version 1.5.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: JudgeNet - the beer judge digest From: David Houseman Reply-To: "JudgeNet - the beer judge digest" Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Subject: Question - Selecting Beers for BOS X-UIDL: 090438977206988 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message I'm sorry but I don't believe one can have an absolute numerical cutoff for qualifying for BOS as Greg suggests. The scoring within any given flight of judges can vary considerably; it's relative to the those judges. While the guidelines [at the bottom of the score sheet] do give a range of scores for GOOD, EXCELLENT, etc., these are in the opinion of the judges scoring. If we were measuring with something totally objective such as a metric ruler or volumetric flask, then, yes, you could have a cut off. However, judging by its nature is subjective. My flight's winner with a score of 32 may be superior to your flight's winner with a score of 39 even if it is the same style. Looking for BOS qualities is even more precarious to make the assumption that numerical values can be used as absolutes. The BUZZ OFF is also a fairly large competition with 501 entries this year. We sent all first place winners to the BOS round including cider and mead (we've had a cider win BOS but then it went on to win cidermaker of the year for the brewer so it really did deserve to win). Five judges judged the 27 or so categories without difficulty and time wasn't all that long although I didn't measure it. I've judged BOS at the Nationals where we had all beer categories. Juding all of them didn't take that much longer than if we'd started with the best 8 because the majority of the time was selecting the winner from the last three. Discarding the first 16 didn't take much time at all. Dave Houseman --Next_Part_SYNC48871D1DB Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Return-Path: Received: from postoffice.harvard.net by synchro.com with POP3 (Mailtraq/1.0.0.997) id SYNC48731C857; Sat, 29 Aug 1998 09:13:17 -0400 Received: from host.warwick.net (host.warwick.net [204.255.24.254]) by deliverance.harvard.net (8.8.7/8.8.7) with ESMTP id JAA20147 for ; Sat, 29 Aug 1998 09:12:12 -0400 (EDT) Received: from warwick.net (w171-23.warwick.net [208.199.17.33]) by host.warwick.net (8.9.1/8.9.1) with ESMTP id IAA28760 for ; Sat, 29 Aug 1998 08:58:10 -0400 (EDT) Message-ID: <35E7FCFF.F255ADE9 at warwick.net> Date: Sat, 29 Aug 1998 09:07:12 -0400 From: David Sherfey Errors-To: judge-owner at synchro.com Sender: judge at synchro.com Reply-To: "JudgeNet - the beer judge digest" X-Mailer: Mozilla 4.04 [en] (Win95; U) MIME-Version: 1.0 To: JudgeNet - the beer judge digest Subject: Re: Selecting Beers for BOS References: Content-Type: text/plain; charset=x-user-defined Content-Transfer-Encoding: 7bit X-UIDL: 090439633320150 X-Hops: 3 X-POST-MessageClass: 10; Mailing List Message Greg-- If a competition informs it's entrants in advance of the BOS policy, it should be OK to handle BOS any way it sees fit. If its a bad policy, the entrants will let the organizers know it by not sending their entries to the competition. IMO, your policy of setting criteria for first place is excellent. I believe that most judges have the ability to determine if an entry is first place or not, and competitions should encourage them to make that call. Just because an entry is the "best" of a flight does not make it a first place beer. There have been occasions in the past where there were barely second place entries in categories I have judged and awarding any of them first place would have sent the wrong message to the brewer. The award is part of the feedback judges give the brewers, and it has to be honest. I believe your policy of limiting the quantity of entries getting to BOS by dropping off the lowest scores is going too far. The limiting criteria for first place should take care of weak entries getting to BOS. All first place entries should proceed to the BOS round. Entrants expect this and especially the individuals working the competition and have beers entered like to see their beers go as far as they can without editorial bias. BOS judges get those extra 1/2 points for that extra effort, and the number of entries they get is just part of the territory! However, since some competitions are held in venues that must be cleared for other activities, the time used at BOS judging needs to be minimized. One way to do this is to split the field in two and have two panels each judge half of the entries and then finish with a taste-off of the best six. Having only five judges makes this a little off center, with an extra judge on one panel. Perhaps the BJCP could allow a change to six judges? I doubt that many judges would pass up the opportunity to judge BOS just because there are 24 entries to judge and they have to drive back to LA or beyond. I judged BOS at one of your past competitions and we still had time to go out for dinner, go over to somone's house for a party, and drive home to LA that night (I drove). It's tough work and we just have to do it. Cheers! David Sherfey Warwick, NY --Next_Part_SYNC48871D1DB--