Return-Path: owner-judge at synchro.com Received: from srvr20.engin.umich.edu (root at srvr20.engin.umich.edu [141.212.2.26]) by srvr5.engin.umich.edu (8.8.5/8.8.5) with ESMTP id KAA03244 for ; Fri, 16 May 1997 10:53:44 -0400 (EDT) Received: from judgmentday.rs.itd.umich.edu (0 at judgmentday.rs.itd.umich.edu [141.211.83.37]) by srvr20.engin.umich.edu (8.8.5/8.8.5) with ESMTP id KAA16138 for ; Fri, 16 May 1997 10:53:43 -0400 (EDT) Received: by judgmentday.rs.itd.umich.edu (8.8.5/2.2) with X.500 id KAA06122; Fri, 16 May 1997 10:53:42 -0400 (EDT) Received: from uu6.psi.com by judgmentday.rs.itd.umich.edu (8.8.5/2.2) with SMTP id KAA06048; Fri, 16 May 1997 10:53:35 -0400 (EDT) Received: by uu6.psi.com (5.65b/4.0.071791-PSI/PSINet) via UUCP; id AA02648 for spencer at umich.edu; Fri, 16 May 97 10:53:01 -0400 Received: (from majordom at localhost) by synchro.com (8.6.12/8.6.12) id KAA06111 for judge-digest-outgoing; Fri, 16 May 1997 10:16:29 -0400 Date: Fri, 16 May 1997 10:16:29 -0400 Message-Id: <199705161416.KAA06111 at synchro.com> From: owner-judge-digest at synchro.com To: judge-digest at synchro.com Subject: judge-digest V1 #1441 Reply-To: judge at synchro.com Errors-To: owner-judge-digest at synchro.com Precedence: bulk judge-digest Friday, 16 May 1997 Volume 01 : Number 1441 ============================================================================ J u d g e N e t - t h e b e e r j u d g e d i g e s t ============================================================================ Moderator: Chuck Cox Archivist: Spencer Thomas Publisher: SynchroSystems Submissions: judge at synchro.com Subscriptions: judge-request at synchro.com Archive: http://realbeer.com/spencer/judge BJCP info: geninfo at bjcp.synchro.com ============================================================================ contents: Mis-categorizing entries porter?/stout? Exam a crapshoot? I beg to differ HOTV competition results returning exam materials NHC Judges Coming to Chicago - Last Minute Instructions ---------------------------------------------------------------------- From: Keith Chamberlin Date: Thu, 15 May 1997 10:13:48 -0400 (EDT) Subject: Mis-categorizing entries I know this may be a bit off topic but I thought I would put it out on the floor and see what develops. I recently entered a belgian tripel in a competition and was surprised I didn't win because it was a wonderful beer, 8.25% abv, very smoothly drinkable, and decieving of the alcohol level, and know real flaws except a little banana. Well the scoresheet came back with a 25 or 28 or something. It was judged as a fruit lambic! All judges commented that it didn't have the fruitiness or sourness. One judge made a comment about alcohol but it was vague so I don't know if that was a shot in the dark or not. One person commented about it being so clear. Why didn't these judges check to see if it was entered in the right category? I don't know, across the board the comments obviously showed it wasn't in the right category yet they didn't do anything about it. Maybe these judges weren't familiar with the style so they let it go, don't know. They only reason this irks me is the fact that in the previous contest I entered I had another beer entered in the wrong category. I had someone else enter it for me at the contest so I can't attest to the accuracy of the actual entry but this Imperial stout, OG 1.098 and black as tar with a black head and almost 9% alcohol was judged as a foreign style stout. Not as far off the mark as the tripel but again, probably could have scored better if judged in the proper category. My question is, what can be done to ensure that the beers are entered properly, short of not allowing judges to bring their entries on the day of? I like entering mine the day of so I know it is handled well and all. In each of these contests I got an award but it would have been nicer to win 2! :) Cheers, Keith ------------------------------ From: popeman at webtv.net (Tom Pope) Date: Thu, 15 May 1997 08:00:06 -0700 Subject: porter?/stout? The porter versus stout issue is complex and interesting, in that stout evolved from porter as a stout porter. Terry Foster gives examples of porter recipes that contain roast barley, and Lewis cites examples of modern commercial stout recipes that contain no roast barley. Old British Beers and how to Make Them lists numerous recipes for old stouts that contained no roast barley and porters that do contain roast barley. It seems to be one of those issues with no black and white answers, but rather one that will call on the combined expertise and judgement of the judges serving on any particular panel. Perhaps we're going back to that thread about judging the beer by the guidelines provided for the particular competition? Cheers.......Tom Pope ------------------------------ From: Scott Bickham Date: Thu, 15 May 1997 11:04:52 -0400 (EDT) Subject: Exam a crapshoot? I beg to differ A study by Dr. (soon to be Prof.) Ed Wolfe has shown that the BJCP exam has great reliability. Ed is a BJCP exam grader with a PhD in educational psychology, with a specialization in the areas of cognitive science and psychometrics. He will soon be leaving his position at the Educational Testing Service (SAT, GRE, etc.) for the University of Florida. Anyway, enough about Ed. To summarize a study which he will submit to a peer-reviewed journal, he determined the following: ...snip... The information above can be used to generate reliability coefficients. Classically, reliability is defined as the ratio of "true variance" to "total variance" where "true variance is the variance attributable only to examinees and total variance is all of the variance in exam scores. There are different ways to define "total variance," and in our case (i.e., when you are distinguishing between "levels" of performance rather than between individual examinees) we should define "total variance" as being the sum of the above components (each divided by the number of components that factors into the final score). If this is too technical, just remember that, in general, reliability coefficients are interpreted as if they are correlation coefficients (i.e., they have a range from 0.00 to 1.00), and levels below 0.50 are (subjectively) considered to be unacceptable, 0.50 to 0.75 are considered to be acceptable, 0.75-0.90 are considered to be good, and 0.90 to 1.00 are considered to be outstanding. Most large-scale essay exams exhibit reliability around 0.80 or so. Anyway, the "index of dependability" (which is the index appropriate for the types of absolute "level" distinctions that we are making) is 0.86 (for an examinee who responds to 10 items, each graded by two graders). This is pretty damned good. It indicates that we can have a lot of confidence in the classifications that we make based on exam scores. ...snip... As the snips indicate, this is just a small part of the preliminary report Ed submitted to the BJCP Board and the exam committee. It is clear that we need to work on standarizing the different exams, since it seems to make a difference which set of questions are given. However very little variance is attributable to the graders, which means that we're being _relatively_similar in our severity when grading exams and that we're not being too "biased" by idiosyncracies of individual examinees. So while the exam is proven to be reliable, we are working to make it even better. Scott Bickham - -- ======================================================================== Naval Research Laboratory, Code 6691 E-mail: bickham at dave.nrl.navy.mil Complex Systems Theory Branch Home or BJCP: 7507 Swan Point Way Washington, DC 20375 Columbia, MD 21045 (202) 404-8632 FAX: (202) 404-7546 (410) 290-7721 BJCP Web Page: http://www.bjcp.org ========================================================================= ------------------------------ From: Mark Taratoot Date: Thu, 15 May 1997 09:33:25 -0700 (PDT) Subject: HOTV competition results This year celebrated the 15th anniversary of the Oregon Homebrewers Competition and Festival. We had a total of 236 entries, over twenty more entries than last year. Thanks to all who participated. Results can be viewed at http://www.peak.org/~taratoot/fest.html - -or- http://www.peak.org/~taratoot/result97.html - -- Mark Taratoot taratoot at peak.org ------------------------------ From: Jeremy Bergsman Date: Thu, 15 May 1997 11:30:49 -0700 Subject: returning exam materials > From: Norman Dickenson > Something bothers me, though, about the public nature > of the appeal, the recipe knowledge and that John received > copies of his scoresheets despite a BJCP policy not to > return exam materials. I agree that such an appeal should not be public (although it seems like it backfired since so many thought it was a robust porter (I told him so in private email)), but I'm concerned about the rest of your statement. Is there really a positive BJCP policy not to return exam materials as opposed to just not having the mechanism/ resources to do so? And what is wrong with his knowing about the recipe? I think one of the big flaws with the exam is the near total lack of feedback. The exam is an incredible learning experience. But the benefit could be improved with better feedback. Many people taking the exam are judging out beers with a score sheet for the first time, or nearly so. Think how much one could learn with rapidly-returned copies of the scoresheet of the proctors and any other examinees who did well. - -- Jeremy Bergsman jeremybb at leland.stanford.edu http://www-leland.stanford.edu/~jeremybb ------------------------------ From: "Roger Deschner " Date: Thu, 15 May 1997 13:05:06 CDT Subject: NHC Judges Coming to Chicago - Last Minute Instructions A big THANK YOU to everyone who signed up to judge at the AHA NHC 1st Round in Chicago. We now appear to have enough judges. Some last-minute instructions for those who are signed up to come: A truck lost its load of a huge chunk of steel-reenforced concrete on the southbound Dan Ryan Expressway (I-90/94) near Comiskey Park last evening, piercing the bridge deck, and partially closing the road. This is causing monumental traffic jams in Chicago, especially on the inbound Kennedy Expressway. It may or may not be fixed by the weekend. When entering the city, tune your radio to AM 780 or AM 670 for traffic reports and the latest information on this mess. Those coming from the north and northwest might do well to enter the city on I-290, instead of I-90/94. PLEASE ALLOW EXTRA TIME FOR WORSE-THAN-USUAL TRAFFIC AS WE NEED TO GET EACH JUDGING SESSION STARTED ON TIME. Routes from the South are not affected by this bridge-piercing. Things to bring: o A sweater. It can be chilly in the brewery. o Your favorite mechanical pencil if you have one. We have some mechanical pencils, but they are the cheap kind. o An eraser. o A stapler if you have one. Put your name on it. o Your best judging palate - we have 825 beers to judge. Send me private e-mail if you need directions or other information. Roger Deschner University of Illinois at Chicago rogerd at uic.edu Aliases: u52983 at uicvm.uic.edu R.Deschner at uic.edu ====Chicago Region Director, 1997 AHA National Homebrew Competitiion==== ------------------------------ End of judge-digest V1 #1441 **************************** Send subscription cancellations & changes to judge-request at synchro.com. Messages sent to the wrong address will be ignored.