Subject: Digest for the period 3/24/2004 - 3/25/2004 Date: Thu, 25 Mar 2004 01:04:33 -0500 Table of contents ---------------------------------------------------------------------- 1. Re: "Language is a virus," exam scoring, imperfect measures (John C.Tull) 2. BJCP exam's the problem, not the path (Tim Oborn) 3. 1. Another exam proposal 2. AHA First round DATES to go with sites? (Bob Paolino) 4. Qualitative v Quantitative knowledge of beers (Jay Hersh aka Dr. Beer(R)) 5. Re:Rote memorization of style guidelines (Tom & Dana Karnowski) 6. new styles (Bryan L. Gros) ---------------------------------------------------------------------- From: John C.Tull Date: Tue, 23 Mar 2004 23:53:52 -0800 Subject: Re: "Language is a virus," exam scoring, imperfect measures On Mar 23, 2004, at 10:01 PM, JudgeNet - the beer judge digest wrote: > The exam graders can't know how "accurate" a proctor's score is, but > iat > least when commercial beers are used they should know what they were to > be able to verify who's "right" when there are big discrepancies in > scoring. The graders of exams are provided with the names of the commercial beers as well as descriptions of their condition, including notable flaws, flavors, smells, etc. that the Exam Administrator detects on any beers. Doctored beers are explained with regard to what was done to them. Usually, a recipe base for the homebrewed beers are included, again with comments on age, etc. I would agree that this is something that needs addressing if it were not already handled about as well as possible under the current system. The better points I have seen on how to help improve this process include standardizing the doctoring process, and working harder to get solid judges to proctor. > also.... > > John C.Tull: > You do not become a National until you have had at least 10 points of > judging experience. This generally represents at least 10 competitions. > Arguably, the tasting skills of that person will improve considerably > after judging 10 competitions, given that there are judges as good or > better than that person on each panel. So National still has to be > earned from some degree of experience regardless of what the score on > the exam is after entering the program. > > That's 20 points (unless what you mean is that at least half of the 20 > points need to be from judging rather than organising, but organiser > points involve only a very small proportion of the membership). My point was that a National judge will have to have a minimum of 10 judging points out of the 20 total. Admin points really are not that hard to come by if you actively organize competitions (don't we all?) . My point is only strengthened by the fact that most people gain their experience points solely from judging. I do not disagree with your point that there are judges with ranks that they never will live up to in the BJCP. These things happen even in the most rigorous examination procedures (there are lots of bad lawyers that did just fine on the bar exam; I also have known lots of graduate students who excelled on the GRE, but can't write and can't design a research project). Cheers, John Tull ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Tim Oborn Date: Wed, 24 Mar 2004 01:08:29 -0700 Subject: BJCP exam's the problem, not the path Fellow Judges, I've been reading all these "alternate paths" comments and I'd like to suggest an "alternative" angle to the arguements; it's not the path that's flawed, IT'S THE TEST!!! I'm a professional educator (I teach chemistry) and I work occasionally developing and marking tests for the Province of Alberta. Without a shadow of a doubt, the exam as it is now is an outdated and archaic form of evaluation. 10 essay questions in 3 hours with tastings? I told my students and colleagues this and they thought it was absolutely crazy. Here are the main problems as I see them: 1) The time limit: We've discussed people who are wonderful judges who can't take tests. I would submit that NO ONE can take this test without some degree of panic, let alone comfort. The stress induced due to the limited time is completely unnecessary and flies in the face of the very spirit of beer judging and appreciation. We are following an insane dogma that originated from a teacher's need to end a class and start another. TIME SHOULD NOT GET IN THE WAY OF SHOWING WHAT YOU KNOW on ANY exam, ideally. Going and gone are the days where teachers rip tests away from a student mid-sentence. Think about this...why SHOULDN'T the examinee be given the time to show what they know? Why are we so bloody worried? Do we as competition organizers rip the exam sheets out of judge's hands? Lengthen the test time to make it reasonable. Or, split the tasting and exam component to be on different days. 2) The structure: Why do all questions need to be essay form? Many of the knowledge based questions can be effectively evaluated using multiple choice, short answer or numerical response. Keep some essay questions, but not more than 3. With this style, you can evaluate a lot more material and in a more efficient way. Secondly, you make the exam marking way less subjective. Despite best intentions, it is very easy to see varying marks given to similar essays (not unlike beers). For example, when hand-picked professional educators mark provincial exams with a set marking key (that we train on for a day), we use 2 markers to evaluate. Despite the skill of the marker, a detailed key and all other checks and balances, the amount of discrepant reads is STILL about 15%. Lastly, a non essay format also has the HUGE added benefit of making the test easier and QUICKER to mark so that a candidate doesn't have to wait..oh, say 5 months for their results (like I did). Another hallmark of a good test: immediate feedback to LEARN from your mistakes. 3) The little silly rules: "5 point deductions" for failure to fill things out completely, zip codes, not putting your registration number exactly and legibly on the sheet, not being allowed to use a laptop to write your answer (why can't a judge evaluate a beer using a laptop?), addition errors...etc. When did we forget that this is a beer judging exam and not the entrance exam into the Hitler Youth? Given the timeframes of 18 minutes per 1-2 full page essay, I wouldn't blame anyone for forgetting these things! "Relax, don't worry, have a homebrew" ...unless you are having a homebrew during a BJCP exam, then it's the extreme opposite. The bottom line is that this exam is the root of most problems. I would guarantee that if you just fixed the time limit problems, you would solve many of the issues and make it more fair to those that truly know their stuff rather than reward those who can write fast and perform under pressure. NO ONE WHO KNOWS ANYTHING ABOUT TEACHING AND LEARNING GIVES THESE TEST PARADIGMS ANYMORE. At the very least, the BJCP should allow candidates to have the option of alternative testing. Remember, this isn't about doing some big bad gauntlet-running test...it's about being given a good and fair opportunity to show one's knowledge and skill. As it stands, the exam is not serving as a fair assessment for a great many individuals. By the way, in case anyone was wondering, I got 80% my first try and this was after brewing for only 3 months. I signed up because my best buddy told me about a 5 month "beer tasting" course. Now, thanks to the course, my buddy and my instructors, I got hooked into homebrewing. However,I couldn't tell a lager from an ale 5 months before I wrote the BJCP exam. In contrast, there were long term brewers in my class who failed yet I know studied their ass off and who, IMHO, knew way more than I did. I knew how to prepare for and take a test. Fair? Thanks for your time, Tim Oborn BSc., B.Ed., B.ottlewasher ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Bob Paolino Date: Wed, 24 Mar 2004 10:17:36 -0500 Subject: 1. Another exam proposal 2. AHA First round DATES to go with sites? I wrote: > scoresheets aren't infallible. One of the proctors for my exam is a > fantastic homebrewer, is generally very perceptive about beer and has a > very gold palate, but (and I hate to say it) a terrible judge. He has > long since stopped judging regularly, having mostly lost interest in The fact that my finger slipped off the "o" to the "l" and gave my homebrewing friend a "gold" palate rather than a "good" palate my provide a mildly ironic context for the proposal I'm about to float. One of the complaints we hear about the exam and how much there is to write in a short time is that people say their writing hands are in physical pain from writing so quickly without rest. Even though one of my colleagues writes everything on legal pads before typing (astonishing to me), most of us probably compose right (write? :-) ) at the keyboard. What would people think about allowing exams to be written on computers? Obviously, you'd not want people to be able to go online or reference their own study materials (don't run them on a network), but most cities in which exams are likely to be given have one of more universities, colleges, or technical schools, and perhaps an exam administrator could arrange to reserve the use of a computer lab (or perhaps a company's training room or other facility with multiple computers). Whether it would allow someone to write faster or a more complete exam is uncertain. I'd certainly be able to write faster _or_ (maybe _and_?) compose my responses better, but I wonder if the ability to edit and rewrite as I go might tempt me to go back and fix minor errors or improve the sentences, thereby offsetting the speed advantage ;-) (although that editing capacity doesn't seem to have helped the gold-palate judges in my emails ;-) ) What is certain, however, is that the exam graders would have a more legible product. Any thoughts? --- Gary wrote: > Interested in judging or stewarding for the National Homebrew > Competition? > Check out www.beertown.org/events/nhc/judging.html for judge > coordinator > contact info for your region. > > First Round Regional judging sites include: > San Diego, CA > Canoga Park, CA > Portland, OR > Seattle, WA > Denver, CO > Basehor, KS > Houston, TX > Chicago, IL > Westlake, OH > Rochester, NY > Rhinebeck, NY (Cider) > > This year's Canadian NHC qualifier is the Ales Home Brew Open in > Regina, SK, > see www.alesclub.com. Do you have the specific judging dates for each site? ZYMURGY and the website don't; they just show April 23 – May 2 rather than specific weekends/days. Thanks. Now go have a beer, Bob Paolino "Homer, why don't you get one of those hands-free phones? It's the next best thing to paying attention to the road." --Barney, The Simpsons ( ) ASCII ribbon campaign X against HTML e-mail: / \ Friends don't send friends HTML-bloated messages! A: Because it reverses the logical flow of conversation. Q: Why is top posting frowned upon? ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Jay Hersh aka Dr. Beer(R) Date: Wed, 24 Mar 2004 17:59:30 -0500 Subject: Qualitative v Quantitative knowledge of beers Well I've held off a bit in chiming in here, but I think some points I've seen here require being addressed. I don't want to single anyone out. I've seen things I agree with mentioned here and things I disagree with. I also don't really want to enter into a point, counter-point situation or an ongoing discussion. So take my comments for what they are worth to you once you understand where they come from and what I base them on. What I base my comments on are my opinions and/or the philosophy I applied during my past tenure as judge (18 yrs, now GM), Exam Director (2.5yrs), Associate Exam Director (2yrs), proctor of 5 exams, grader of hundreds of tests, and reviewer of the grading of hundreds more. I can tell you that at least during my tenure (and I believe prior to and since) that the policy of the BJCP when grading exams is that in order to score at the highest levels you must demonstrate BOTH Qualitative and Quantitative knowledge of the beers. I agree with the assertion that it is important to learn the terminology and language associated with beer and brewing. But it is as equally important to learn the analytical aspects of describing beers. The BJCP exam is, has been, and I hope will continue to be a measure of BOTH of these skills. To my knowledge, and during my tenure as administrator, persons who might be able to recite a table of analytical descriptors for a style, but couldn't accompany that analytical knowledge with the ability to communicate in terms of the type of descriptive terminology you cite, scored in the same class as those who could not recite the analytics but could be descriptive. Such individuals virtually always found themselves scoring in the high 70s or low 80s. The ranks of 90+ were routinely populated by those that had (appropriately in my opinion) grasped both skills. The ranks of the 60s were usually populated by people who demonstrated sufficient knowledge to demonstrate they had some skills but needed to develop their skills further. I think there is and continues to be misunderstanding among many participants here of what the exam is meant to be, and in fact what any exam is even possible of assessing. The exam is not, nor was it ever meant to be, a perfect assessment of one's judging skills. It is a test of knowledge sufficient to insure some minimum standard of competency among people judging at competitions. It has never asserted to be a guarantee that any judge, at any level is completely familiar with every aspect of every style. While it is more unlikely that someone attaining a score of 90+ on the exam will have significant gaps in their knowledge there is no way a single 3 hour exam can insure this. The proportion of those taking the BJCP exam that fail is pretty small. I think most people reading here would admit that with reasonable preparation it is not too difficult to pass the exam. My personal experience is that those who did not pass either significantly misjudged the material and thus didn't properly prepare, or just plain didn't bother to prepare. The materials currently available on the BJCP site are in my opinion sufficient to assist one in preparing so as to gain a passing score. The BJCP recognizes that this is a hobby. It doesn't set out looking to fail people. Rather it looks to assess the knowledge an examinee demonstrates, insure that it meets a minimum standard for us to approve, and encourage membership and advancement. We have always tried to facilitate education. It's a lot tougher than you think to prepare good materials, and find and train teachers. I myself have developed (and made freely available) my trademarked Dr. Beer sensory training guidelines for many years and have used them to train many judges. I've also helped others to use these too. Many others have developed similar materials. It has taken a bit of time to accomplish this, especially given the fact that as the program has grown, the chorus of differing opinions grows and concensus seems to be harder to easily achieve. Conversely the proportion of those taking the exam that score highly is also rather small. It is relatively rare that first time takers score over a 90. I've seen some do it, and the one individual I know who achieved this wrote a test as good as any I've ever seen. This is however by design not accident. Being a program meant to challenge it's members to continue to improve themselves and to help others to improve themselves, the exam, while easy to pass, does require a strong knowledge of the Qualitative and Quantitative aspects of beer and brewing. I think this is appropriate. The program anticipates that those achieving the highest levels within it should be those with as broad and deep a knowledge of the material as possible. This is because it is these same people that the program hopes will then use that level of knowledge to train others that follow after them. Along these lines the program has structured itself to recognize that volunteer service in the form of developing educational materials the program uses, proctoring exams, grading exams, and helping administer the program as part of the overall process of moving up to it's highest ranks. Again this is nothing I see as being inappropriate. If we don't create incentives for people to conduct the day to day operations of the organization, and the ongoing training of new members, then how can we expect new members to be able to advance to a point where they can teach others and run the program when the current generation begins to hand things off? In short I don't think new ideas are bad. I've seen some here I've liked and would consider implementing if we have the resources. But I think that the fundamental structure of the BJCP is sound. I say this because over the course of 18 years I have already seen the program undergo a painful transformation and come out stronger. In the last few years I've witnessed new members come in and add a lot of value to the program. And I know that some of the same things people are most upset with are actively being addressed, but I'd advise being patient. From the long term perspective I take this organization has come a long way and is in my opinion as strong as it has ever been. Jay Hersh ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Tom & Dana Karnowski Date: Wed, 24 Mar 2004 22:17:48 -0500 Subject: Re:Rote memorization of style guidelines Bill Pierce discusses quantitative vs qualitative descriptions of beer styles. Although I am somewhat scientific myself, so maybe I"m biased, I don't think Bickham was being too quantitative when he said "Beer styles are characterized by numerical ranges for many characteristics, so how can one accurately evaluate those styles if he or she cannot give ballpark figures for those attributes?" The key point here is ballpark. Personally I think a description like Bill's is fine. "Oktoberfest as a medium to dark amber beer of moderate gravity and not excessive bitterness, with the emphasis on malt in the flavor profile." In fact I think it is MUCH better than just saying "the color should be 7-14 SRM, the O.G. 1.050-1.064 and the bittering 20-30 IBUs". But what is even better than both is the following: "Oktoberfest as a medium to dark amber beer of moderate gravity and not excessive bitterness, with the emphasis on malt in the flavor profile. OG approximately 1.050-1.060, around 25 IBU, and color about 7-10 SRM." And if you added the fact they are lager beers, made with pilsner malt and munich malt, hopped with noble hops, and add in some historical context and commercial examples too, you've just taken the cake. The numbers are not the be-all to end-all. But they COMPLIMENT the descriptions that Bill uses and truly illustrate what you mean by "medium to dark amber", "moderate gravity", and "not excessive bitterness". And I think, as such, they are indispensible in showing your knowledge of a beer style. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Bryan L. Gros Date: Wed, 24 Mar 2004 19:59:33 -0800 Subject: new styles Saw on the AHA site where the forthcoming BJCP style guidelines will include a few new styles, including Baltic Porter and Double IPA Double IPA is a popular brewpub and homebrew style, at least out here in California, and I can see a number of entries there in a competition. But can any committee members comment on why Baltic Porter has been added? Unless it is some advance take on Double or Imperial porter, I don't see brewers that I know making anything like it, nor any need for it at any regional competitions. Is it something the AHA wanted? What commercial examples are readily available? I know that the last revision the group recognized the aim of the guidelines was strictly to support homebrew competitions, and therefore Dutch Pilsner dropped off, since there wasn't much demand. Bryan Gros bgros`at`aggienetwork.com Oakland, CA Draught Board Homebrew Club http://www.draughtboard.org ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** Subject: Digest for the period 3/24/2004 - 3/25/2004 Date: Thu, 25 Mar 2004 01:04:33 -0500 Table of contents ---------------------------------------------------------------------- 1. Re: "Language is a virus," exam scoring, imperfect measures (John C.Tull) 2. BJCP exam's the problem, not the path (Tim Oborn) 3. 1. Another exam proposal 2. AHA First round DATES to go with sites? (Bob Paolino) 4. Qualitative v Quantitative knowledge of beers (Jay Hersh aka Dr. Beer(R)) 5. Re:Rote memorization of style guidelines (Tom & Dana Karnowski) 6. new styles (Bryan L. Gros) ---------------------------------------------------------------------- From: John C.Tull Date: Tue, 23 Mar 2004 23:53:52 -0800 Subject: Re: "Language is a virus," exam scoring, imperfect measures On Mar 23, 2004, at 10:01 PM, JudgeNet - the beer judge digest wrote: > The exam graders can't know how "accurate" a proctor's score is, but > iat > least when commercial beers are used they should know what they were to > be able to verify who's "right" when there are big discrepancies in > scoring. The graders of exams are provided with the names of the commercial beers as well as descriptions of their condition, including notable flaws, flavors, smells, etc. that the Exam Administrator detects on any beers. Doctored beers are explained with regard to what was done to them. Usually, a recipe base for the homebrewed beers are included, again with comments on age, etc. I would agree that this is something that needs addressing if it were not already handled about as well as possible under the current system. The better points I have seen on how to help improve this process include standardizing the doctoring process, and working harder to get solid judges to proctor. > also.... > > John C.Tull: > You do not become a National until you have had at least 10 points of > judging experience. This generally represents at least 10 competitions. > Arguably, the tasting skills of that person will improve considerably > after judging 10 competitions, given that there are judges as good or > better than that person on each panel. So National still has to be > earned from some degree of experience regardless of what the score on > the exam is after entering the program. > > That's 20 points (unless what you mean is that at least half of the 20 > points need to be from judging rather than organising, but organiser > points involve only a very small proportion of the membership). My point was that a National judge will have to have a minimum of 10 judging points out of the 20 total. Admin points really are not that hard to come by if you actively organize competitions (don't we all?) . My point is only strengthened by the fact that most people gain their experience points solely from judging. I do not disagree with your point that there are judges with ranks that they never will live up to in the BJCP. These things happen even in the most rigorous examination procedures (there are lots of bad lawyers that did just fine on the bar exam; I also have known lots of graduate students who excelled on the GRE, but can't write and can't design a research project). Cheers, John Tull ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Tim Oborn Date: Wed, 24 Mar 2004 01:08:29 -0700 Subject: BJCP exam's the problem, not the path Fellow Judges, I've been reading all these "alternate paths" comments and I'd like to suggest an "alternative" angle to the arguements; it's not the path that's flawed, IT'S THE TEST!!! I'm a professional educator (I teach chemistry) and I work occasionally developing and marking tests for the Province of Alberta. Without a shadow of a doubt, the exam as it is now is an outdated and archaic form of evaluation. 10 essay questions in 3 hours with tastings? I told my students and colleagues this and they thought it was absolutely crazy. Here are the main problems as I see them: 1) The time limit: We've discussed people who are wonderful judges who can't take tests. I would submit that NO ONE can take this test without some degree of panic, let alone comfort. The stress induced due to the limited time is completely unnecessary and flies in the face of the very spirit of beer judging and appreciation. We are following an insane dogma that originated from a teacher's need to end a class and start another. TIME SHOULD NOT GET IN THE WAY OF SHOWING WHAT YOU KNOW on ANY exam, ideally. Going and gone are the days where teachers rip tests away from a student mid-sentence. Think about this...why SHOULDN'T the examinee be given the time to show what they know? Why are we so bloody worried? Do we as competition organizers rip the exam sheets out of judge's hands? Lengthen the test time to make it reasonable. Or, split the tasting and exam component to be on different days. 2) The structure: Why do all questions need to be essay form? Many of the knowledge based questions can be effectively evaluated using multiple choice, short answer or numerical response. Keep some essay questions, but not more than 3. With this style, you can evaluate a lot more material and in a more efficient way. Secondly, you make the exam marking way less subjective. Despite best intentions, it is very easy to see varying marks given to similar essays (not unlike beers). For example, when hand-picked professional educators mark provincial exams with a set marking key (that we train on for a day), we use 2 markers to evaluate. Despite the skill of the marker, a detailed key and all other checks and balances, the amount of discrepant reads is STILL about 15%. Lastly, a non essay format also has the HUGE added benefit of making the test easier and QUICKER to mark so that a candidate doesn't have to wait..oh, say 5 months for their results (like I did). Another hallmark of a good test: immediate feedback to LEARN from your mistakes. 3) The little silly rules: "5 point deductions" for failure to fill things out completely, zip codes, not putting your registration number exactly and legibly on the sheet, not being allowed to use a laptop to write your answer (why can't a judge evaluate a beer using a laptop?), addition errors...etc. When did we forget that this is a beer judging exam and not the entrance exam into the Hitler Youth? Given the timeframes of 18 minutes per 1-2 full page essay, I wouldn't blame anyone for forgetting these things! "Relax, don't worry, have a homebrew" ...unless you are having a homebrew during a BJCP exam, then it's the extreme opposite. The bottom line is that this exam is the root of most problems. I would guarantee that if you just fixed the time limit problems, you would solve many of the issues and make it more fair to those that truly know their stuff rather than reward those who can write fast and perform under pressure. NO ONE WHO KNOWS ANYTHING ABOUT TEACHING AND LEARNING GIVES THESE TEST PARADIGMS ANYMORE. At the very least, the BJCP should allow candidates to have the option of alternative testing. Remember, this isn't about doing some big bad gauntlet-running test...it's about being given a good and fair opportunity to show one's knowledge and skill. As it stands, the exam is not serving as a fair assessment for a great many individuals. By the way, in case anyone was wondering, I got 80% my first try and this was after brewing for only 3 months. I signed up because my best buddy told me about a 5 month "beer tasting" course. Now, thanks to the course, my buddy and my instructors, I got hooked into homebrewing. However,I couldn't tell a lager from an ale 5 months before I wrote the BJCP exam. In contrast, there were long term brewers in my class who failed yet I know studied their ass off and who, IMHO, knew way more than I did. I knew how to prepare for and take a test. Fair? Thanks for your time, Tim Oborn BSc., B.Ed., B.ottlewasher ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Bob Paolino Date: Wed, 24 Mar 2004 10:17:36 -0500 Subject: 1. Another exam proposal 2. AHA First round DATES to go with sites? I wrote: > scoresheets aren't infallible. One of the proctors for my exam is a > fantastic homebrewer, is generally very perceptive about beer and has a > very gold palate, but (and I hate to say it) a terrible judge. He has > long since stopped judging regularly, having mostly lost interest in The fact that my finger slipped off the "o" to the "l" and gave my homebrewing friend a "gold" palate rather than a "good" palate my provide a mildly ironic context for the proposal I'm about to float. One of the complaints we hear about the exam and how much there is to write in a short time is that people say their writing hands are in physical pain from writing so quickly without rest. Even though one of my colleagues writes everything on legal pads before typing (astonishing to me), most of us probably compose right (write? :-) ) at the keyboard. What would people think about allowing exams to be written on computers? Obviously, you'd not want people to be able to go online or reference their own study materials (don't run them on a network), but most cities in which exams are likely to be given have one of more universities, colleges, or technical schools, and perhaps an exam administrator could arrange to reserve the use of a computer lab (or perhaps a company's training room or other facility with multiple computers). Whether it would allow someone to write faster or a more complete exam is uncertain. I'd certainly be able to write faster _or_ (maybe _and_?) compose my responses better, but I wonder if the ability to edit and rewrite as I go might tempt me to go back and fix minor errors or improve the sentences, thereby offsetting the speed advantage ;-) (although that editing capacity doesn't seem to have helped the gold-palate judges in my emails ;-) ) What is certain, however, is that the exam graders would have a more legible product. Any thoughts? --- Gary wrote: > Interested in judging or stewarding for the National Homebrew > Competition? > Check out www.beertown.org/events/nhc/judging.html for judge > coordinator > contact info for your region. > > First Round Regional judging sites include: > San Diego, CA > Canoga Park, CA > Portland, OR > Seattle, WA > Denver, CO > Basehor, KS > Houston, TX > Chicago, IL > Westlake, OH > Rochester, NY > Rhinebeck, NY (Cider) > > This year's Canadian NHC qualifier is the Ales Home Brew Open in > Regina, SK, > see www.alesclub.com. Do you have the specific judging dates for each site? ZYMURGY and the website don't; they just show April 23 – May 2 rather than specific weekends/days. Thanks. Now go have a beer, Bob Paolino "Homer, why don't you get one of those hands-free phones? It's the next best thing to paying attention to the road." --Barney, The Simpsons ( ) ASCII ribbon campaign X against HTML e-mail: / \ Friends don't send friends HTML-bloated messages! A: Because it reverses the logical flow of conversation. Q: Why is top posting frowned upon? ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Jay Hersh aka Dr. Beer(R) Date: Wed, 24 Mar 2004 17:59:30 -0500 Subject: Qualitative v Quantitative knowledge of beers Well I've held off a bit in chiming in here, but I think some points I've seen here require being addressed. I don't want to single anyone out. I've seen things I agree with mentioned here and things I disagree with. I also don't really want to enter into a point, counter-point situation or an ongoing discussion. So take my comments for what they are worth to you once you understand where they come from and what I base them on. What I base my comments on are my opinions and/or the philosophy I applied during my past tenure as judge (18 yrs, now GM), Exam Director (2.5yrs), Associate Exam Director (2yrs), proctor of 5 exams, grader of hundreds of tests, and reviewer of the grading of hundreds more. I can tell you that at least during my tenure (and I believe prior to and since) that the policy of the BJCP when grading exams is that in order to score at the highest levels you must demonstrate BOTH Qualitative and Quantitative knowledge of the beers. I agree with the assertion that it is important to learn the terminology and language associated with beer and brewing. But it is as equally important to learn the analytical aspects of describing beers. The BJCP exam is, has been, and I hope will continue to be a measure of BOTH of these skills. To my knowledge, and during my tenure as administrator, persons who might be able to recite a table of analytical descriptors for a style, but couldn't accompany that analytical knowledge with the ability to communicate in terms of the type of descriptive terminology you cite, scored in the same class as those who could not recite the analytics but could be descriptive. Such individuals virtually always found themselves scoring in the high 70s or low 80s. The ranks of 90+ were routinely populated by those that had (appropriately in my opinion) grasped both skills. The ranks of the 60s were usually populated by people who demonstrated sufficient knowledge to demonstrate they had some skills but needed to develop their skills further. I think there is and continues to be misunderstanding among many participants here of what the exam is meant to be, and in fact what any exam is even possible of assessing. The exam is not, nor was it ever meant to be, a perfect assessment of one's judging skills. It is a test of knowledge sufficient to insure some minimum standard of competency among people judging at competitions. It has never asserted to be a guarantee that any judge, at any level is completely familiar with every aspect of every style. While it is more unlikely that someone attaining a score of 90+ on the exam will have significant gaps in their knowledge there is no way a single 3 hour exam can insure this. The proportion of those taking the BJCP exam that fail is pretty small. I think most people reading here would admit that with reasonable preparation it is not too difficult to pass the exam. My personal experience is that those who did not pass either significantly misjudged the material and thus didn't properly prepare, or just plain didn't bother to prepare. The materials currently available on the BJCP site are in my opinion sufficient to assist one in preparing so as to gain a passing score. The BJCP recognizes that this is a hobby. It doesn't set out looking to fail people. Rather it looks to assess the knowledge an examinee demonstrates, insure that it meets a minimum standard for us to approve, and encourage membership and advancement. We have always tried to facilitate education. It's a lot tougher than you think to prepare good materials, and find and train teachers. I myself have developed (and made freely available) my trademarked Dr. Beer sensory training guidelines for many years and have used them to train many judges. I've also helped others to use these too. Many others have developed similar materials. It has taken a bit of time to accomplish this, especially given the fact that as the program has grown, the chorus of differing opinions grows and concensus seems to be harder to easily achieve. Conversely the proportion of those taking the exam that score highly is also rather small. It is relatively rare that first time takers score over a 90. I've seen some do it, and the one individual I know who achieved this wrote a test as good as any I've ever seen. This is however by design not accident. Being a program meant to challenge it's members to continue to improve themselves and to help others to improve themselves, the exam, while easy to pass, does require a strong knowledge of the Qualitative and Quantitative aspects of beer and brewing. I think this is appropriate. The program anticipates that those achieving the highest levels within it should be those with as broad and deep a knowledge of the material as possible. This is because it is these same people that the program hopes will then use that level of knowledge to train others that follow after them. Along these lines the program has structured itself to recognize that volunteer service in the form of developing educational materials the program uses, proctoring exams, grading exams, and helping administer the program as part of the overall process of moving up to it's highest ranks. Again this is nothing I see as being inappropriate. If we don't create incentives for people to conduct the day to day operations of the organization, and the ongoing training of new members, then how can we expect new members to be able to advance to a point where they can teach others and run the program when the current generation begins to hand things off? In short I don't think new ideas are bad. I've seen some here I've liked and would consider implementing if we have the resources. But I think that the fundamental structure of the BJCP is sound. I say this because over the course of 18 years I have already seen the program undergo a painful transformation and come out stronger. In the last few years I've witnessed new members come in and add a lot of value to the program. And I know that some of the same things people are most upset with are actively being addressed, but I'd advise being patient. From the long term perspective I take this organization has come a long way and is in my opinion as strong as it has ever been. Jay Hersh ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Tom & Dana Karnowski Date: Wed, 24 Mar 2004 22:17:48 -0500 Subject: Re:Rote memorization of style guidelines Bill Pierce discusses quantitative vs qualitative descriptions of beer styles. Although I am somewhat scientific myself, so maybe I"m biased, I don't think Bickham was being too quantitative when he said "Beer styles are characterized by numerical ranges for many characteristics, so how can one accurately evaluate those styles if he or she cannot give ballpark figures for those attributes?" The key point here is ballpark. Personally I think a description like Bill's is fine. "Oktoberfest as a medium to dark amber beer of moderate gravity and not excessive bitterness, with the emphasis on malt in the flavor profile." In fact I think it is MUCH better than just saying "the color should be 7-14 SRM, the O.G. 1.050-1.064 and the bittering 20-30 IBUs". But what is even better than both is the following: "Oktoberfest as a medium to dark amber beer of moderate gravity and not excessive bitterness, with the emphasis on malt in the flavor profile. OG approximately 1.050-1.060, around 25 IBU, and color about 7-10 SRM." And if you added the fact they are lager beers, made with pilsner malt and munich malt, hopped with noble hops, and add in some historical context and commercial examples too, you've just taken the cake. The numbers are not the be-all to end-all. But they COMPLIMENT the descriptions that Bill uses and truly illustrate what you mean by "medium to dark amber", "moderate gravity", and "not excessive bitterness". And I think, as such, they are indispensible in showing your knowledge of a beer style. ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * ********************************************************************** ---------------------------------------------------------------------- From: Bryan L. Gros Date: Wed, 24 Mar 2004 19:59:33 -0800 Subject: new styles Saw on the AHA site where the forthcoming BJCP style guidelines will include a few new styles, including Baltic Porter and Double IPA Double IPA is a popular brewpub and homebrew style, at least out here in California, and I can see a number of entries there in a competition. But can any committee members comment on why Baltic Porter has been added? Unless it is some advance take on Double or Imperial porter, I don't see brewers that I know making anything like it, nor any need for it at any regional competitions. Is it something the AHA wanted? What commercial examples are readily available? I know that the last revision the group recognized the aim of the guidelines was strictly to support homebrew competitions, and therefore Dutch Pilsner dropped off, since there wasn't much demand. Bryan Gros bgros`at`aggienetwork.com Oakland, CA Draught Board Homebrew Club http://www.draughtboard.org ********************************************************************** * JudgeNet - the beer judge digest * * Send plain text only, no HTML, MIME, encoded text or attachments * * Send subscription requests & changes to judge-request`at`synchro.com * **********************************************************************