Subject: Digest for the period 1/16/2004 - 1/17/2004 Date: Sat, 17 Jan 2004 01:01:46 -0500 Table of contents ---------------------------------------------------------------------- 1. Re: BJCP exam heresy (Scott Bickham) 2. RE: BJCP Exam Heresy (Houseman, David L) 3. RE: BJCP Exam Heresy (Jamil Zainasheff) 4. Re: Digest for the period 1/15/2004 - 1/16/2004 (Mark Wilson) ---------------------------------------------------------------------- From: Scott Bickham Date: Fri, 16 Jan 2004 06:53:08 -0500 Subject: Re: BJCP exam heresy > Dion asks - > "Can anyone give me a valid reason for requiring rote memorization of the > BJCP Style Guidelines in order to pass the exam? To me, this seems > unrealistic. Since at every judging we are given the complete style > guidelines, what is the benefit of having to memorize them?" As other posters have pointed out, this assumption is laregly incorrect. While I agree that most graders will look for approximate numerical ranges in the "perfect" answers, they are not a part of most answers that earn as many as 8 or 9 points on a 10 point scale. A number of 80+ scores have been given out on exams without a single IBU or OG range. This aspect of a beer description has never been a requirement for passing the exam. So why can numerical ranges help earn higher scores? Because in competitions and tastings, judges often have to make qualitative comparisons of similar beers, and this is much more accurate when at least approximate characteristics are known. As an example, here are two levels of comments which are quite common on competition scoresheets. Judge A. This beer is too hoppy. Judge B. This beer has a pronounced hop bitterness than is higher than the upper end of the 20-30 IBU range given in the style guidelines. I think it's pretty clear that the brewer who paid to have his beer judged would be much more appreciative of the feedback given by Judge B. A similar argument can be used to show how this type of data can quantify comparisons, as opposed to just saying that one beer is hoppier or more alcoholic than another, for example. If anyone feels that he or she did not pass the exam primarily on the basis of not giving numerical ranges, please let me know. Good brewing, Scott ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Houseman, David L Date: Fri, 16 Jan 2004 09:00:11 -0500 Subject: RE: BJCP Exam Heresy I can't comment on how all graders grade the exam (there is a guide) but as I grade, and have helped numerous new graders get started, it is not a matter, as Greg points out, that we [should] count nits by looking at exact numbers to match the published specs, but rather how the individual seems to understand the styles. I don't know of any exam question where one needed to know that Category 15 was Porter! What is asked, in a number of ways, is to describe and differentiate styles. By describe what is being asked is "what are the characteristics (aroma, appearance, flavor, mouthfeel, overall) that make up this style." By differentiate, how do the stated styles differ or are similar. Giving the specs (at least approximately) of OG, IBU, SRM, alc/vol, indicates a greater depth of knowledge. If a judge didn't know these things for the essay portion of the exam, how would there be an expectation that they could judge adequately on the tasting portion of the exam? Sure, there usually are copies of the style guide available to judges in competitions, but when you have to judge flights at 10min a beer, that is not the time to be reading and learning about the style. We all may refresh our memories at that point, but the better judges largely know the styles going in. That's what differentiates the levels of the judges. I, like many, had to take the exam twice to get 90 or better, and I'm probably worse at memorizing facts than most. But I found that creating a set of flash cards to study really helped. While I have forgot many of the details, I think the experience was worth the effort and I now have a deeper understanding of the styles by studying sufficiently to achieve the higher score. I'd wager anyone on this list could as well. Dave Houseman ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Jamil Zainasheff Date: Fri, 16 Jan 2004 07:50:12 -0800 Subject: RE: BJCP Exam Heresy Let's say a person were able to memorize every last number from the guidelines, that isn't enough to pass the exam at even a Recognized level. And numbers alone do not make the difference between Certified and National or any other level. The difference between the various levels is not a rote memorization of the numbers, but rather a broader understanding of every aspect of a beer style including brewing, history, taste, etc. Yes, it can be very challenging to communicate that breadth of knowledge in 3 hours. I'm with Greg. The examinee can demonstrate that they have a detailed and well read understanding of a style, without perfectly repeating the numbers from the guidelines. For example, if the upper limit for IPA is 1.075 and the examinee lists 1.060, perhaps they're confused with pale ale and don't really understand the style? If instead they say it is around 1.070 and go on to explain how the higher hopping and OG helped protect the beer on its travel to India, then it is clear they have some breadth of knowledge and a better understanding of the style. JZ > I personally > don't think that anyone needs to demonstrate "exact" memorization of the > guideline statistics to demonstrate superior knowledge. But as a grader, > I'm looking for a "close" approximation of the more important statistics > (OG, IBUs, and SRM). If an exam-taker can right down statistics that are > close to the "real" numbers, then they are demonstrating a depth of > knowledge that would translate to a high grade (other parts of the answer > being equally high, of course). If you tell me that the OG of an > Oktoberfest was 1.048 to 1.060 (guidelines say 1.050 to 1.064), I would > probably give you full credit, assuming that the rest of your > description of > the style was consistent with the style. If you said that the OG of a > barley wine was 1.040 to 1.060, then I would probably grade you > down for it. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Mark Wilson Date: Fri, 16 Jan 2004 10:58:21 -0800 Subject: Re: Digest for the period 1/15/2004 - 1/16/2004 > > I think your question relates to variability among graders rather than an > overall formal expectation that a National (or Master) judge must be able to > rattle off stats without the benefit of style guides at hand. I personally > don't think that anyone needs to demonstrate "exact" memorization of the > guideline statistics to demonstrate superior knowledge. But as a grader, > I'm looking for a "close" approximation of the more important statistics > (OG, IBUs, and SRM). If an exam-taker can right down statistics that are Well, this gets at one of MY pet peeves about the exam. There is no "grader's guide" availible, so exam-takers don't know how close you need to be, and what exactly points are being taken off for. Am I penalized for being off spec by 1 point? 3? 5? Since threshold, for example, for IBU perception is about 5 IBUs, I hope not. Not to mention the fact that many BJCP classic examples do not meet the style parameters (e.g. Old Peculiar). Should one concentrate on memorizing these sort of details, or more history, geography, etc. in order to improve their score? It would be really great if it were possible to return the exams with grader comments, so one could improve for next time. I fully understand WHY this is not the case (nobody would grade exams, lest they be constantly barraged by whining exams takers), but the amount of feedback you currently get is very low. More examples of correct answers would also be helpful. ----------------------------------------------------------------- Mark W. Wilson mwilson`at`agora.rdrop.com Portland, OR http://www.rdrop.com/users/mwilson ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** Subject: Digest for the period 1/16/2004 - 1/17/2004 Date: Sat, 17 Jan 2004 01:01:46 -0500 Table of contents ---------------------------------------------------------------------- 1. Re: BJCP exam heresy (Scott Bickham) 2. RE: BJCP Exam Heresy (Houseman, David L) 3. RE: BJCP Exam Heresy (Jamil Zainasheff) 4. Re: Digest for the period 1/15/2004 - 1/16/2004 (Mark Wilson) ---------------------------------------------------------------------- From: Scott Bickham Date: Fri, 16 Jan 2004 06:53:08 -0500 Subject: Re: BJCP exam heresy > Dion asks - > "Can anyone give me a valid reason for requiring rote memorization of the > BJCP Style Guidelines in order to pass the exam? To me, this seems > unrealistic. Since at every judging we are given the complete style > guidelines, what is the benefit of having to memorize them?" As other posters have pointed out, this assumption is laregly incorrect. While I agree that most graders will look for approximate numerical ranges in the "perfect" answers, they are not a part of most answers that earn as many as 8 or 9 points on a 10 point scale. A number of 80+ scores have been given out on exams without a single IBU or OG range. This aspect of a beer description has never been a requirement for passing the exam. So why can numerical ranges help earn higher scores? Because in competitions and tastings, judges often have to make qualitative comparisons of similar beers, and this is much more accurate when at least approximate characteristics are known. As an example, here are two levels of comments which are quite common on competition scoresheets. Judge A. This beer is too hoppy. Judge B. This beer has a pronounced hop bitterness than is higher than the upper end of the 20-30 IBU range given in the style guidelines. I think it's pretty clear that the brewer who paid to have his beer judged would be much more appreciative of the feedback given by Judge B. A similar argument can be used to show how this type of data can quantify comparisons, as opposed to just saying that one beer is hoppier or more alcoholic than another, for example. If anyone feels that he or she did not pass the exam primarily on the basis of not giving numerical ranges, please let me know. Good brewing, Scott ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Houseman, David L Date: Fri, 16 Jan 2004 09:00:11 -0500 Subject: RE: BJCP Exam Heresy I can't comment on how all graders grade the exam (there is a guide) but as I grade, and have helped numerous new graders get started, it is not a matter, as Greg points out, that we [should] count nits by looking at exact numbers to match the published specs, but rather how the individual seems to understand the styles. I don't know of any exam question where one needed to know that Category 15 was Porter! What is asked, in a number of ways, is to describe and differentiate styles. By describe what is being asked is "what are the characteristics (aroma, appearance, flavor, mouthfeel, overall) that make up this style." By differentiate, how do the stated styles differ or are similar. Giving the specs (at least approximately) of OG, IBU, SRM, alc/vol, indicates a greater depth of knowledge. If a judge didn't know these things for the essay portion of the exam, how would there be an expectation that they could judge adequately on the tasting portion of the exam? Sure, there usually are copies of the style guide available to judges in competitions, but when you have to judge flights at 10min a beer, that is not the time to be reading and learning about the style. We all may refresh our memories at that point, but the better judges largely know the styles going in. That's what differentiates the levels of the judges. I, like many, had to take the exam twice to get 90 or better, and I'm probably worse at memorizing facts than most. But I found that creating a set of flash cards to study really helped. While I have forgot many of the details, I think the experience was worth the effort and I now have a deeper understanding of the styles by studying sufficiently to achieve the higher score. I'd wager anyone on this list could as well. Dave Houseman ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Jamil Zainasheff Date: Fri, 16 Jan 2004 07:50:12 -0800 Subject: RE: BJCP Exam Heresy Let's say a person were able to memorize every last number from the guidelines, that isn't enough to pass the exam at even a Recognized level. And numbers alone do not make the difference between Certified and National or any other level. The difference between the various levels is not a rote memorization of the numbers, but rather a broader understanding of every aspect of a beer style including brewing, history, taste, etc. Yes, it can be very challenging to communicate that breadth of knowledge in 3 hours. I'm with Greg. The examinee can demonstrate that they have a detailed and well read understanding of a style, without perfectly repeating the numbers from the guidelines. For example, if the upper limit for IPA is 1.075 and the examinee lists 1.060, perhaps they're confused with pale ale and don't really understand the style? If instead they say it is around 1.070 and go on to explain how the higher hopping and OG helped protect the beer on its travel to India, then it is clear they have some breadth of knowledge and a better understanding of the style. JZ > I personally > don't think that anyone needs to demonstrate "exact" memorization of the > guideline statistics to demonstrate superior knowledge. But as a grader, > I'm looking for a "close" approximation of the more important statistics > (OG, IBUs, and SRM). If an exam-taker can right down statistics that are > close to the "real" numbers, then they are demonstrating a depth of > knowledge that would translate to a high grade (other parts of the answer > being equally high, of course). If you tell me that the OG of an > Oktoberfest was 1.048 to 1.060 (guidelines say 1.050 to 1.064), I would > probably give you full credit, assuming that the rest of your > description of > the style was consistent with the style. If you said that the OG of a > barley wine was 1.040 to 1.060, then I would probably grade you > down for it. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Mark Wilson Date: Fri, 16 Jan 2004 10:58:21 -0800 Subject: Re: Digest for the period 1/15/2004 - 1/16/2004 > > I think your question relates to variability among graders rather than an > overall formal expectation that a National (or Master) judge must be able to > rattle off stats without the benefit of style guides at hand. I personally > don't think that anyone needs to demonstrate "exact" memorization of the > guideline statistics to demonstrate superior knowledge. But as a grader, > I'm looking for a "close" approximation of the more important statistics > (OG, IBUs, and SRM). If an exam-taker can right down statistics that are Well, this gets at one of MY pet peeves about the exam. There is no "grader's guide" availible, so exam-takers don't know how close you need to be, and what exactly points are being taken off for. Am I penalized for being off spec by 1 point? 3? 5? Since threshold, for example, for IBU perception is about 5 IBUs, I hope not. Not to mention the fact that many BJCP classic examples do not meet the style parameters (e.g. Old Peculiar). Should one concentrate on memorizing these sort of details, or more history, geography, etc. in order to improve their score? It would be really great if it were possible to return the exams with grader comments, so one could improve for next time. I fully understand WHY this is not the case (nobody would grade exams, lest they be constantly barraged by whining exams takers), but the amount of feedback you currently get is very low. More examples of correct answers would also be helpful. ----------------------------------------------------------------- Mark W. Wilson mwilson`at`agora.rdrop.com Portland, OR http://www.rdrop.com/users/mwilson ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * **********************************************************************