Subject: Digest for the period 3/22/2004 - 3/23/2004 Date: Tue, 23 Mar 2004 01:02:22 -0500 Table of contents ---------------------------------------------------------------------- 1. Clarification (Jeremy Bergsman) 2. Comments in Favor of the Alternate Path to Master Judge (Greg Lorton) 3. Re: Competiton Organizers Reports (Ed Westemeier) 4. Re: Alternate path to rank (John C.Tull) 5. Re: Proposal (John C.Tull) 6. Reporting of AHA comps (Gordon Strong) 7. AHA 1st Round-Great Lakes-Call for Judges & Stewards (Joe Preiser) 8. Rote memorization and exam score (Scott Bickham) ---------------------------------------------------------------------- From: Jeremy Bergsman Date: Sun, 21 Mar 2004 09:06:57 -0500 Subject: Clarification > From: Bill Wible > Jeremy Bergsman writes: > > >Experience points are already in the equation. Likewise, > >no "service" (e.g. exam grading) should substitute. The > >value of the exam is that it is an independent measure > >of a judge's skill--as opposed to participation. > > I couldn't disagree more, and I'm sure others are going > to say the same thing. > > 1) There's nothing "independent" about the current exam. I apologize for not making myself clear. By "independent" I meant independent from experience points. I did not mean to imply that it was therefore comprehensive. My argument is that there are currently two measures being taken to determine judge rank: test score and experience. These are essentially independent measures in the following way: The exam measures "book learning" and the ability to physically write coherent answers to essay questions in a relatively short period of time, and judge 4 beers without the style guide present. Experience points measure the number of times you sit down to judge at competitions, regardless of the quality of your work, the number of beers in the flight, the style, etc. (and service activities). Obviously both measures are imperfect, which gives some value to retaining both of them. The briefly instantiated alternate path to master simply offered the chance to lessen one measure and increase the other. My proposal was to allow the possibility to partially substitute for exam score a third independent measure--again imperfect--educational activities. I think this is better than simply relying more heavily on one of the measures. By describing points as a measure of participation I only mean to present the worst case scenario--like much of the discussion around the exam. Of course an interested and prepared person will learn a great deal from each flight of beers he/she judges. This is why experience should count. But it should not be the be-all and end-all, just like the exam shouldn't. I hope with this explanation that you see that I wasn't defending the test (as you saw in my specific proposal I suggested changing the test as the first, most important step). However I would like to respond to some of your points anyway. > proctor giving it, (who decides on the beers and also > scores them with you. We've had NUMEROUS complaints > on that front, where a proctor has given one of his > own beers as a test beer, rated it highly, and it wasn't > that good of a beer. This is not supposed to be how it works. But I agree that the current system is not sufficiently stringent: those tasting the beer as proctors should have ZERO knowledge of the beer other than what the examinees have and should not be allowed to confer. This has been the case at the exams I have participated in. > The questions on each version of > the test are different, so it's not even the "same" > test from version to version. This is true, but what would you do instead? This is also true of the SAT, the bar exam, and every test you took in high school and college. In fact, the exam is incredibly similar from test to test, other than the styles involved. Most of the styles used are ones that should be familiar to every judge. The graders also take into account the difficulty of the questions in their grading. > 2) Some people are very knowledgeable about beer and > styles, and can talk your ear off and know what they're > talking about, but just cannot take a test. Agreed. What is your solution to this problem? > Why not just make someone a master judge > as soon as they score over a 90, if the exam score is all > that matters? > > So NO, I don't the BJCP exam in and of itself is the > be all and end all of a measure of a judge's knowledge. > And it can't be used that way. I hope I've made it clear that this was my point. In fact I said exactly this in the first sentence of my post in Friday's digest. -- Jeremy Bergsman jeremy`at`bergsman.org http://www.bergsman.org/jeremy ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Greg Lorton Date: Sun, 21 Mar 2004 11:18:12 -0800 Subject: Comments in Favor of the Alternate Path to Master Judge In the most recent JudgeNets, it seems to me that there is a shift of opinion away from the proposal by Mike Hall for an alternate path to Master judge level. So I guess I better get my comments in now (in favor of Mike's original proposal). :-) I believe that there is a rather fuzzy line between the National and Master level in terms of the essay portion of exam. According to the exam scoring guidelines, a National level essay should have "Good knowledge of all subjects. Some errors allowable, but no significant gaps. Some depth indicated." For the Master level, the exam should have "Excellent knowledge level. No significant errors, and no gaps. Good depth to answers." I believe that much of this swings on the expectation of "depth" by the graders, and this seems subjective to me. Some graders expect to read a historical description of the beers (even though the style questions do not explicitly ask for the history, or even hint at it, only the exam scoring guidelines do). I took the exam again after becoming a lead grader, and I believe I provided a very solid essay, writing non-stop for three hours, minus the time to evaluate the four beers. (I thought I nailed it!) :-) However, my score was in the upper 80s because I didn't elaborate on the historical aspects of the beers in the style questions (e.g., impact of taxation in England on the strength of stouts). On the tasting side, I think a problem that needs to be overcome is the quality control aspects of it, particularly by the proctors. I graded an exam not too long ago where the proctors were Certified judges and the exam scores of several examinees were at the National and Master level. A number of the examinees identified flaws that the person who prepared the beers said were there, but that the proctors missed. As a second issue on the tasting, I believe that the proctors should provide two scores for each beer, one based on their individual evaluations, and one that reflects the score after they have discussed the beers. I believe this would better address insensitivities that the proctors may have to certain characteristics (e.g., I am pretty insensitive to diacetyl, and find it appealing at levels that are obnoxious to judges who are sensitive to it). In the exam that I recently graded, one proctor gave a beer a 30, while the other gave it a 45. Which score do you use to compare with the examinees? (the average, or just one of the two) Most of the examinees gave the beer scores that centered around the 30. It is difficult for an examinee to get a good tasting score if there is wide variability in the scores and comments provided by the proctors. It's hard to decide who's right! The guidelines presented in an earlier JudgeNet for preparing the beers for the exam should be followed (i.e, an exemplary example of the style; a good beer, but out of style; a doctored beer; etc.) Having high level, experience judges serve as proctors seems necessary, but apparently isn't always doable. Finally, I agree that no judge should give up judging because he or she sees no opportunity for advancement. On the other hand, grading exams does seem like a rather thankless job when the incentives for doing it benefit only Master judges in the long run. Cheers, Greg Lorton ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Ed Westemeier Date: Sun, 21 Mar 2004 18:25:25 -0500 Subject: Re: Competiton Organizers Reports On 3/21/04, Mike Dixon said: >How will the reporting work with comps that are registered through the >AHA? >Our competition is next weekend and the organizer did it through the AHA >will he still be able to send the BJCP report online? Think about what you're reporting -- BJCP judging and staff points. The AHA doesn't care about them, so use the new BJCP online system. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: John C.Tull Date: Sun, 21 Mar 2004 15:41:14 -0800 Subject: Re: Alternate path to rank Mark Vernon made some points that I would like to address. I have abbreviated his original comments to limit the length of my post. On Mar 19, 2004, at 10:04 PM, Mark Vernon wrote: > 1. Several people have put forth that allowing people to up their test > score by using points could allow bad judges to advance. What makes you > think that a bad judge could not score well on the test? ... Anyone > with the time > can memorize most (if not all) of the info on the written portion - so > you get an 86 on the written and a 70 on the taste...viola 81 and you > are National..... You do not become a National until you have had at least 10 points of judging experience. This generally represents at least 10 competitions. Arguably, the tasting skills of that person will improve considerably after judging 10 competitions, given that there are judges as good or better than that person on each panel. So National still has to be earned from some degree of experience regardless of what the score on the exam is after entering the program. > 2. As others have said - lets not limit this to Master level, this > should be an option for any level I agree that any alternate pathway for advancement should not be limited to the privileged few. I also do not believe that we should have an alternate pathway. > 3. ...Well when I took the test (2 yrs ago) I received a 76 on the > written and the comments > specifically said - you need to show the numbers when comparing > styles....and I was not the only one that saw that....so maybe the > graders are not following the guidelines. All I know is that virtually > every judge that I talk to says they also believe you must memorize the > styles to get a high (80+) score. As a grader, I am not concerned with rote memorization from anyone. Number ranges that mostly overlap a style are very beneficial, but an examinee can convey the important information for a style qualitatively without present numbers. For example, if someone describes a Berliner Weiss as "a beer of excpetionally light body and color, sometimes slightly cloudy, with very little alcohol; hops are virtually non-existant", I know that they understand the vital characteristics of a Berliner Weiss. If they provide a great description of the style with history and other elements of depth, they are going to get a 90+ score from me. With that said, numbers written down on paper are easier to examine as a grader, but they are not necessary. > > > Mark Vernon - Certified with points to spare (if the BJCP ever catches > up on their back log!!!) The back log will be remedied in short order. John C. Tull BJCP National Reno, NV P.S. I like Jeremy Bergsman's ideas posted in the same digest with regard to advancement. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: John C.Tull Date: Sun, 21 Mar 2004 15:51:56 -0800 Subject: Re: Proposal On Mar 20, 2004, at 10:01 PM, Bill Wibble wrote: > 1) There's nothing "independent" about the current exam. > That's the basis of the complaints. The entire exam is > variable, from the questions on each version, to the > proctor giving it, (who decides on the beers and also > scores them with you. We've had NUMEROUS complaints > on that front, where a proctor has given one of his > own beers as a test beer, rated it highly, and it wasn't > that good of a beer. The questions on each version of > the test are different, so it's not even the "same" > test from version to version. Who has been administering exams in your area? If your proctor is also providing beer, you are not receiving the exam according to how it was designed to be administered. Under no circumstance should a proctor provide the administrator with beers. The proctors are specifically to have no more knowledge about the beer being served than any of the examinees. Proctors are to evaluate beers independently with no capability of revising their sheets. If it has been done in a manner as you described, you need to report this incident, and the administrator of the exam should not be allowed to administer in the future. Cheers, John C. Tull BJCP National Reno, NV ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Gordon Strong Date: Sun, 21 Mar 2004 23:32:50 -0500 Subject: Reporting of AHA comps To answer Mike Dixon's question about reporting AHA-sanctioned competitions using the new BJCP organizer reporting system, yes, any competition registered with either the AHA or the BJCP can have its points recorded using the web-based system. As a status update, we are now using this same system as a way of distributing the data entry duties to work through the backlog of reports. We'll publish an update to the judge record web site around the end of the month that shows the updated judge records and the current data entry backlog. If any organizer who has submitted a paper report (but who has not seen the points recorded yet) wants to use the new online system to help us work through the stack, please contact the BJCP Program Administrator (program_admin`at`bjcp.org). We just need to check that the report hasn't already been entered, and that the organizer has the necessary access information to submit the report electronically. The list of reports that have been received, recorded, or are delinquent are available in the Reports and Demographics section of the BJCP Judge Records web site (accessed via the main BJCP web site). The data published on the site is about three weeks old, so there are quite a few pending changes. Gordon Strong BJCP IT Director ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Joe Preiser Date: Mon, 22 Mar 2004 12:57:39 -1000 Subject: AHA 1st Round-Great Lakes-Call for Judges & Stewards Great Lakes - NHC 1st Round Call for Judges & Stewards Judging for the Great Lakes region (MI, IL, WI, IN) of the NHC 1st Round will be held at Goose Island-Wrigleyville (3535 N. Clark, Chicago). Please note that this is a change in venue from previous years. Judging sessions are scheduled for Saturday and Sunday, May 1-2. Both judges (primarily BJCP, but others will be considered) and stewards (experienced or not) are needed. Sessions will begin promptly at 9:00AM on Saturday and 10:00AM on Sunday. Stewards should arrive between 8:00AM and 8:15AM to help get things set up. Judges should plan to arrive 30 minutes early for sign-in, bagels, juice, coffee, and judging assignments. If enough judges sign up for Saturday's sessions and we get everything judged we won't schedule a Sunday session; however, this means we need every able-bodied palate we can get. In other words, yes, we can use you. Interested judges and stewards can register online at http://www.chibeer.org/NHC/ or can contact Judging Coordinator, Joe Preiser (joe`at`chibeer.org, daytime phone 630-285-7688). ******************************************************************** You are subscribed to the Urban Knaves of Grain mailing list. To post a message to the list, please send it to ukg`at`hbd.org. To unsubscribe, send the word "unsubscribe" as the body of a message to ukg-request`at`hbd.org. For a list of other useful commands, send the word "help" to ukg-request`at`hbd.org. Comments or questions related to the operation of this list should be directed to ukg-owner`at`hbd.org. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Scott Bickham Date: Mon, 22 Mar 2004 22:28:25 -0500 Subject: Rote memorization and exam score Mark Vernon wrote: " Now I know that some out there > are saying "But you don't have to memorize things" Well when I took > the test (2 yrs ago) I received a 76 on the written and the comments > specifically said - you need to show the numbers when comparing > styles....and I was not the only one that saw that....so maybe the > graders are not following the guidelines. All I know is that virtually > every judge that I talk to says they also believe you must memorize > the styles to get a high (80+) score. I'm glad you brought this up. I think this is one of the key misunderstandings of this topic. I do not deny that that comment was part of Mark's feedback, but showing the numbers is entirely different than rote memorization. It's probably something that needs to be made more clear in the exam feedback. Maybe if we take a step back and consider that one of our key roles as beer judges is to compare the beer being judged against the style guidelines. Our customer is the brewer, who either followed a recipe or formaluted a recipe to target a given style. If the beer is off the mark in some respect, say hop bitterness, then one degree of feedback is to say the beer is too hoppy for style. But the more experienced or advanced judges can probably determine some approximate, quantitative information about the beer, e.g. whether it has 10 IBUs (just above threshold), 20, 30, 40 or 50 IBUs. As a brewer, it is more helpful to read that you think a recipe needs 5-10 IBUs rather than just say that it needs more hop bitterness. Would I expect a certified judge to give that detailed feedback? No. But I would from a National or Master judge, and many brewers specifically seek out competitions where they know their entries will get that feedback level. Back to Mark's exam. The feedback he got on his exam was not a request to memorize the style guidelines, but more of a recommendation to at least attempt to approximately define stylistic differences. One certainly should not be asked to do this to pass the exam, or score above 70, but I think it's a fair request for someone aspiring to be a National or Master judge. Beer styles are characterized by numerical ranges for many characteristics, so how can one accurately evaluate those styles if he or she cannot give ballpark figures for those attributes? These attributes are also key factors in distingushing between similar styles, but the exam does not test whether you have memorized the style guidelines, but whether you understand these differences. Scott Bickham ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** Subject: Digest for the period 3/22/2004 - 3/23/2004 Date: Tue, 23 Mar 2004 01:02:22 -0500 Table of contents ---------------------------------------------------------------------- 1. Clarification (Jeremy Bergsman) 2. Comments in Favor of the Alternate Path to Master Judge (Greg Lorton) 3. Re: Competiton Organizers Reports (Ed Westemeier) 4. Re: Alternate path to rank (John C.Tull) 5. Re: Proposal (John C.Tull) 6. Reporting of AHA comps (Gordon Strong) 7. AHA 1st Round-Great Lakes-Call for Judges & Stewards (Joe Preiser) 8. Rote memorization and exam score (Scott Bickham) ---------------------------------------------------------------------- From: Jeremy Bergsman Date: Sun, 21 Mar 2004 09:06:57 -0500 Subject: Clarification > From: Bill Wible > Jeremy Bergsman writes: > > >Experience points are already in the equation. Likewise, > >no "service" (e.g. exam grading) should substitute. The > >value of the exam is that it is an independent measure > >of a judge's skill--as opposed to participation. > > I couldn't disagree more, and I'm sure others are going > to say the same thing. > > 1) There's nothing "independent" about the current exam. I apologize for not making myself clear. By "independent" I meant independent from experience points. I did not mean to imply that it was therefore comprehensive. My argument is that there are currently two measures being taken to determine judge rank: test score and experience. These are essentially independent measures in the following way: The exam measures "book learning" and the ability to physically write coherent answers to essay questions in a relatively short period of time, and judge 4 beers without the style guide present. Experience points measure the number of times you sit down to judge at competitions, regardless of the quality of your work, the number of beers in the flight, the style, etc. (and service activities). Obviously both measures are imperfect, which gives some value to retaining both of them. The briefly instantiated alternate path to master simply offered the chance to lessen one measure and increase the other. My proposal was to allow the possibility to partially substitute for exam score a third independent measure--again imperfect--educational activities. I think this is better than simply relying more heavily on one of the measures. By describing points as a measure of participation I only mean to present the worst case scenario--like much of the discussion around the exam. Of course an interested and prepared person will learn a great deal from each flight of beers he/she judges. This is why experience should count. But it should not be the be-all and end-all, just like the exam shouldn't. I hope with this explanation that you see that I wasn't defending the test (as you saw in my specific proposal I suggested changing the test as the first, most important step). However I would like to respond to some of your points anyway. > proctor giving it, (who decides on the beers and also > scores them with you. We've had NUMEROUS complaints > on that front, where a proctor has given one of his > own beers as a test beer, rated it highly, and it wasn't > that good of a beer. This is not supposed to be how it works. But I agree that the current system is not sufficiently stringent: those tasting the beer as proctors should have ZERO knowledge of the beer other than what the examinees have and should not be allowed to confer. This has been the case at the exams I have participated in. > The questions on each version of > the test are different, so it's not even the "same" > test from version to version. This is true, but what would you do instead? This is also true of the SAT, the bar exam, and every test you took in high school and college. In fact, the exam is incredibly similar from test to test, other than the styles involved. Most of the styles used are ones that should be familiar to every judge. The graders also take into account the difficulty of the questions in their grading. > 2) Some people are very knowledgeable about beer and > styles, and can talk your ear off and know what they're > talking about, but just cannot take a test. Agreed. What is your solution to this problem? > Why not just make someone a master judge > as soon as they score over a 90, if the exam score is all > that matters? > > So NO, I don't the BJCP exam in and of itself is the > be all and end all of a measure of a judge's knowledge. > And it can't be used that way. I hope I've made it clear that this was my point. In fact I said exactly this in the first sentence of my post in Friday's digest. -- Jeremy Bergsman jeremy`at`bergsman.org http://www.bergsman.org/jeremy ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Greg Lorton Date: Sun, 21 Mar 2004 11:18:12 -0800 Subject: Comments in Favor of the Alternate Path to Master Judge In the most recent JudgeNets, it seems to me that there is a shift of opinion away from the proposal by Mike Hall for an alternate path to Master judge level. So I guess I better get my comments in now (in favor of Mike's original proposal). :-) I believe that there is a rather fuzzy line between the National and Master level in terms of the essay portion of exam. According to the exam scoring guidelines, a National level essay should have "Good knowledge of all subjects. Some errors allowable, but no significant gaps. Some depth indicated." For the Master level, the exam should have "Excellent knowledge level. No significant errors, and no gaps. Good depth to answers." I believe that much of this swings on the expectation of "depth" by the graders, and this seems subjective to me. Some graders expect to read a historical description of the beers (even though the style questions do not explicitly ask for the history, or even hint at it, only the exam scoring guidelines do). I took the exam again after becoming a lead grader, and I believe I provided a very solid essay, writing non-stop for three hours, minus the time to evaluate the four beers. (I thought I nailed it!) :-) However, my score was in the upper 80s because I didn't elaborate on the historical aspects of the beers in the style questions (e.g., impact of taxation in England on the strength of stouts). On the tasting side, I think a problem that needs to be overcome is the quality control aspects of it, particularly by the proctors. I graded an exam not too long ago where the proctors were Certified judges and the exam scores of several examinees were at the National and Master level. A number of the examinees identified flaws that the person who prepared the beers said were there, but that the proctors missed. As a second issue on the tasting, I believe that the proctors should provide two scores for each beer, one based on their individual evaluations, and one that reflects the score after they have discussed the beers. I believe this would better address insensitivities that the proctors may have to certain characteristics (e.g., I am pretty insensitive to diacetyl, and find it appealing at levels that are obnoxious to judges who are sensitive to it). In the exam that I recently graded, one proctor gave a beer a 30, while the other gave it a 45. Which score do you use to compare with the examinees? (the average, or just one of the two) Most of the examinees gave the beer scores that centered around the 30. It is difficult for an examinee to get a good tasting score if there is wide variability in the scores and comments provided by the proctors. It's hard to decide who's right! The guidelines presented in an earlier JudgeNet for preparing the beers for the exam should be followed (i.e, an exemplary example of the style; a good beer, but out of style; a doctored beer; etc.) Having high level, experience judges serve as proctors seems necessary, but apparently isn't always doable. Finally, I agree that no judge should give up judging because he or she sees no opportunity for advancement. On the other hand, grading exams does seem like a rather thankless job when the incentives for doing it benefit only Master judges in the long run. Cheers, Greg Lorton ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Ed Westemeier Date: Sun, 21 Mar 2004 18:25:25 -0500 Subject: Re: Competiton Organizers Reports On 3/21/04, Mike Dixon said: >How will the reporting work with comps that are registered through the >AHA? >Our competition is next weekend and the organizer did it through the AHA >will he still be able to send the BJCP report online? Think about what you're reporting -- BJCP judging and staff points. The AHA doesn't care about them, so use the new BJCP online system. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: John C.Tull Date: Sun, 21 Mar 2004 15:41:14 -0800 Subject: Re: Alternate path to rank Mark Vernon made some points that I would like to address. I have abbreviated his original comments to limit the length of my post. On Mar 19, 2004, at 10:04 PM, Mark Vernon wrote: > 1. Several people have put forth that allowing people to up their test > score by using points could allow bad judges to advance. What makes you > think that a bad judge could not score well on the test? ... Anyone > with the time > can memorize most (if not all) of the info on the written portion - so > you get an 86 on the written and a 70 on the taste...viola 81 and you > are National..... You do not become a National until you have had at least 10 points of judging experience. This generally represents at least 10 competitions. Arguably, the tasting skills of that person will improve considerably after judging 10 competitions, given that there are judges as good or better than that person on each panel. So National still has to be earned from some degree of experience regardless of what the score on the exam is after entering the program. > 2. As others have said - lets not limit this to Master level, this > should be an option for any level I agree that any alternate pathway for advancement should not be limited to the privileged few. I also do not believe that we should have an alternate pathway. > 3. ...Well when I took the test (2 yrs ago) I received a 76 on the > written and the comments > specifically said - you need to show the numbers when comparing > styles....and I was not the only one that saw that....so maybe the > graders are not following the guidelines. All I know is that virtually > every judge that I talk to says they also believe you must memorize the > styles to get a high (80+) score. As a grader, I am not concerned with rote memorization from anyone. Number ranges that mostly overlap a style are very beneficial, but an examinee can convey the important information for a style qualitatively without present numbers. For example, if someone describes a Berliner Weiss as "a beer of excpetionally light body and color, sometimes slightly cloudy, with very little alcohol; hops are virtually non-existant", I know that they understand the vital characteristics of a Berliner Weiss. If they provide a great description of the style with history and other elements of depth, they are going to get a 90+ score from me. With that said, numbers written down on paper are easier to examine as a grader, but they are not necessary. > > > Mark Vernon - Certified with points to spare (if the BJCP ever catches > up on their back log!!!) The back log will be remedied in short order. John C. Tull BJCP National Reno, NV P.S. I like Jeremy Bergsman's ideas posted in the same digest with regard to advancement. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: John C.Tull Date: Sun, 21 Mar 2004 15:51:56 -0800 Subject: Re: Proposal On Mar 20, 2004, at 10:01 PM, Bill Wibble wrote: > 1) There's nothing "independent" about the current exam. > That's the basis of the complaints. The entire exam is > variable, from the questions on each version, to the > proctor giving it, (who decides on the beers and also > scores them with you. We've had NUMEROUS complaints > on that front, where a proctor has given one of his > own beers as a test beer, rated it highly, and it wasn't > that good of a beer. The questions on each version of > the test are different, so it's not even the "same" > test from version to version. Who has been administering exams in your area? If your proctor is also providing beer, you are not receiving the exam according to how it was designed to be administered. Under no circumstance should a proctor provide the administrator with beers. The proctors are specifically to have no more knowledge about the beer being served than any of the examinees. Proctors are to evaluate beers independently with no capability of revising their sheets. If it has been done in a manner as you described, you need to report this incident, and the administrator of the exam should not be allowed to administer in the future. Cheers, John C. Tull BJCP National Reno, NV ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Gordon Strong Date: Sun, 21 Mar 2004 23:32:50 -0500 Subject: Reporting of AHA comps To answer Mike Dixon's question about reporting AHA-sanctioned competitions using the new BJCP organizer reporting system, yes, any competition registered with either the AHA or the BJCP can have its points recorded using the web-based system. As a status update, we are now using this same system as a way of distributing the data entry duties to work through the backlog of reports. We'll publish an update to the judge record web site around the end of the month that shows the updated judge records and the current data entry backlog. If any organizer who has submitted a paper report (but who has not seen the points recorded yet) wants to use the new online system to help us work through the stack, please contact the BJCP Program Administrator (program_admin`at`bjcp.org). We just need to check that the report hasn't already been entered, and that the organizer has the necessary access information to submit the report electronically. The list of reports that have been received, recorded, or are delinquent are available in the Reports and Demographics section of the BJCP Judge Records web site (accessed via the main BJCP web site). The data published on the site is about three weeks old, so there are quite a few pending changes. Gordon Strong BJCP IT Director ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Joe Preiser Date: Mon, 22 Mar 2004 12:57:39 -1000 Subject: AHA 1st Round-Great Lakes-Call for Judges & Stewards Great Lakes - NHC 1st Round Call for Judges & Stewards Judging for the Great Lakes region (MI, IL, WI, IN) of the NHC 1st Round will be held at Goose Island-Wrigleyville (3535 N. Clark, Chicago). Please note that this is a change in venue from previous years. Judging sessions are scheduled for Saturday and Sunday, May 1-2. Both judges (primarily BJCP, but others will be considered) and stewards (experienced or not) are needed. Sessions will begin promptly at 9:00AM on Saturday and 10:00AM on Sunday. Stewards should arrive between 8:00AM and 8:15AM to help get things set up. Judges should plan to arrive 30 minutes early for sign-in, bagels, juice, coffee, and judging assignments. If enough judges sign up for Saturday's sessions and we get everything judged we won't schedule a Sunday session; however, this means we need every able-bodied palate we can get. In other words, yes, we can use you. Interested judges and stewards can register online at http://www.chibeer.org/NHC/ or can contact Judging Coordinator, Joe Preiser (joe`at`chibeer.org, daytime phone 630-285-7688). ******************************************************************** You are subscribed to the Urban Knaves of Grain mailing list. To post a message to the list, please send it to ukg`at`hbd.org. To unsubscribe, send the word "unsubscribe" as the body of a message to ukg-request`at`hbd.org. For a list of other useful commands, send the word "help" to ukg-request`at`hbd.org. Comments or questions related to the operation of this list should be directed to ukg-owner`at`hbd.org. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Scott Bickham Date: Mon, 22 Mar 2004 22:28:25 -0500 Subject: Rote memorization and exam score Mark Vernon wrote: " Now I know that some out there > are saying "But you don't have to memorize things" Well when I took > the test (2 yrs ago) I received a 76 on the written and the comments > specifically said - you need to show the numbers when comparing > styles....and I was not the only one that saw that....so maybe the > graders are not following the guidelines. All I know is that virtually > every judge that I talk to says they also believe you must memorize > the styles to get a high (80+) score. I'm glad you brought this up. I think this is one of the key misunderstandings of this topic. I do not deny that that comment was part of Mark's feedback, but showing the numbers is entirely different than rote memorization. It's probably something that needs to be made more clear in the exam feedback. Maybe if we take a step back and consider that one of our key roles as beer judges is to compare the beer being judged against the style guidelines. Our customer is the brewer, who either followed a recipe or formaluted a recipe to target a given style. If the beer is off the mark in some respect, say hop bitterness, then one degree of feedback is to say the beer is too hoppy for style. But the more experienced or advanced judges can probably determine some approximate, quantitative information about the beer, e.g. whether it has 10 IBUs (just above threshold), 20, 30, 40 or 50 IBUs. As a brewer, it is more helpful to read that you think a recipe needs 5-10 IBUs rather than just say that it needs more hop bitterness. Would I expect a certified judge to give that detailed feedback? No. But I would from a National or Master judge, and many brewers specifically seek out competitions where they know their entries will get that feedback level. Back to Mark's exam. The feedback he got on his exam was not a request to memorize the style guidelines, but more of a recommendation to at least attempt to approximately define stylistic differences. One certainly should not be asked to do this to pass the exam, or score above 70, but I think it's a fair request for someone aspiring to be a National or Master judge. Beer styles are characterized by numerical ranges for many characteristics, so how can one accurately evaluate those styles if he or she cannot give ballpark figures for those attributes? These attributes are also key factors in distingushing between similar styles, but the exam does not test whether you have memorized the style guidelines, but whether you understand these differences. Scott Bickham ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * **********************************************************************