From JudgeNet Mon Mar 2 13:15:48 1992 Subject: Re: BJCP Exam Study Guide (2 Reply to: RE>BJCP Exam Study Guide (2/29 Chuck; Outstanding! I hope you did OK on the exam yesterday. From the guide, I suspect you did quite well... On the question of commercial examples which are not available in the US, how about listing such in parenthesis or brackets? Then those who do get to Europe will be able to take advantage of your experience as well. Also it never hurts to know some European examples to list in your description of a style. This is especially helpful where there is no US version of a style. I think as long as you identify these names you won't confuse the examinees. RW... -------------------------------------- From JudgeNet Mon Mar 2 18:14:23 1992 Subject: Physiology of Tasting Good job, Chuck, hope you enjoyed the exam on Sunday (or at least did really well). One thing that has struck me as I study for the BJCP exam is the apparent lack of discussion about the act of tasting the beer itself. Rather it is the beer that is studied. Shouldn't judges know about sensory levels, ppb sensitivity, threshold levels of chemicals, taste areas of the tongue, effect of temperature and carbonation levels on certain flavors, etc.? Am I missing something basic here? Should there be questions devoted solely to *how* people taste beers? Just wondering. Thanks in advance for replies. Internet: jdecarlo at mitre.org (or John.DeCarlo at f131.n109.z1.fidonet.org) Fidonet: 1:109/131 -------------------------------------- From JudgeNet Mon Mar 2 18:19:08 1992 Subject: Study Guide Addendum Subject: Study Guide Addendum Time:12:44 PM Date:3/2/92 In reading through the Guide prior to printing it out I noticed that there is no mention of the four beers which will be judged during the exam. I thought the following, added to the Introduction would cover the topic. "You will also be asked to judge (4) beers as a part of the exam. These beers will be presented during the time you are taking the written portion, spaced about 20 minutes apart. The styles will be identified for you by the examiner. Judge these beers as you would for a competition using the AHA score sheet. Be clear and consise in your comments. Identify any off aromas and flavors you perceive. Suggest possible causes for these problems. Legibility is also an important factor, if your comments cannot be read they won't convey your impressions to the brewer or the person grading the exam. 30% of your final score will be determined by this portion of the exam." RW... Russ Wigglesworth CI$: 72300,61 |~~| UCSF Medical Center Internet: Rad Equipment at RadMac1.ucsf.edu |HB|\ Dept. of Radiology, Rm. C-324 Voice: 415-476-3668 / 474-8126 (H) |__|/ San Francisco, CA 94143-0628 -------------------------------------- From JudgeNet Mon Mar 2 18:29:08 1992 Subject: Re: Study Guide comments - style section I wanted to respond a little more thoroughly to Chuck and to Steve Stroud but my editor keeps scrambling the long lines. So briefly: I agree with Steve that the French ales should be kept to themselves. Most of the Bieres de garde I've tasted have been pretty abused, but they are as worthy of their own categorization as any of the Belgian styles. Widmer is a good example of an alt, at least compared to anything else available in the US. I've tasted both this and the Zum Uerlige (spelling suspect); Widmer has reduced their intense bitterness from the original (and will eventually return the recipe to the truer version) but it's still a very close match. Martin's Pale Ale belong with British pales: the local distributor/importer sez its the same as Courage Bulldog. Douglas Scotch Ale is simply a higher gravity version of McEwan's and belongs with the Scotch Ales. Unless someone can convince me otherwise, I'd say that "stock ale", at least as far as Sam Adams is concerned, is a bogus category. It belongs with the American pale ales. --Jeff Frane -------------------------------------- From JudgeNet Wed Mar 4 17:14:15 1992 Subject: Scoring Guide We are getting ready to train our next generation of perspective judges for the Ithaca Brewers' Union so we wanted to outline a scoring procedure for the new judges to start with. I have written a procedure down, and would like to get some feedback from this list for improvements to it. My goal in mind here was to put write down some scoring realities which are not really presented to the new judge in other areas. Remember this is an outline trying to teach a new judge how to score, some of it is my personal preference, but I've tried to outline a procedure which is a start, providing a steeping stone for a judge developing his/her own methods. Thanks in Advance for any input. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc3.bitnet _____________________________________ Suggested Evaluation Procedure What follows is a suggested evaluation procedure for evaluating the adherence of a particular beer to style and to identify flaws present in a beer relative to that style. As a new judge, you can use this to help you define your own scoring methods, to help make sure your scores are on target with your judging partner. Helpful additional reading in this area is the AHA national competition judging manual. The competition sheet has 5 scoring areas with a certain number of points available in each area and a maximum total score of 50 points. The governing factor in how you score beers is the Scoring Guide given at the bottom of the sheet: Excellent 40-50 An Excellent representation of style Very Good 30-39 A very good representation of style, has minor flaw(s) Good 25-29 A good representation of style, has significant flaw(s) Drinkable 20-24 A drinkable beer, but serious flaw(s) distinguish it from the style Problem 0-19 A beer with serious flaw(s) which prevent it from being drinkable, regardless of style. Some general practical scoring notes: No real beer is a 50. The best beers at a competition will score in the 40-47 realm, with real evaluations almost always revealing some point about the beer which is not perfect, even with the best commercial examples (e.g. no beer will be perfectly fresh). When you evaluate a beer, try to pick out something in the beer which is correct, and make sure you mention it to the brewer (the appearance, color, body). This helps make the criticism which will follow easier to swallow, and probably causes the brewer to have more faith in your evaluation. No beer deserves below a 19. In general, when you send a beer to a contest, the brewer has paid money to get a helpful, unbiased evaluation of a beer. Everyone who has made a beer has made one which is not up to par, so why scar a brewer's scoresheet with poor marks and ill-chosen, unhelpful comments? Find places to give a problem beer points which total 19, even if they wouldn't normally be deserved. This lets the brewer know the beer needs attention, which is all that needs to be conveyed. Any serious style flaw in a beer will drop it to around 30. For example, a Bavarian wheat beer without clove character, a bock of conventional strength, or a dry stout without roasted barley would all have a maximum score of around 30. The scoring procedure: 1. (If given the bottle) Check the bottle for fill level, clarity, sediment, as well as a ring around the neck at the fill level, and note results. This is useful to alert you of flaws which may be in the beer, but be careful not to prejudge the beer. 2. Smell the aroma immediately after the beer is poured into the glass, and write down the components of what you're smelling. 3. Score the appearance of the beer: a) Clarity: 2 pts - brilliant, 0 pts - cloudy, 1 pt - in between b) Color: 2 pts - to style, 1 pt - light or dark, 0 pts - way off c) Head Retention 2 pts - Very Good, 1 pt - Good, 0 pts - nonexistent Note on a), don't forget the clarity of a beer is style dependent. Note on c), be careful to gauge head relative to the carbonation level in the beer. 4. Return to the aroma of the beer after giving your nose a rest, and write down full notes on the aroma, and how it changed from (2). Do NOT score the aroma yet. 5. Taste the beer's flavor profile a) initially - malt, hops, alcohol, sweetness, spiciness b) intermediate - additional hop/malt flavor, fruitiness, diacetyl, sourness c) aftertaste - hop bitterness, oxidation, astringency d) conditioning - appropriate carbonation level for style, how this carbonation affects the flavor perception of the beer - does it come out all at once? not at all? Note on a) to c) - these are examples of what you could taste, you would write down all that you taste. Do NOT score the flavor yet. 6. Score the Body (viscosity, mouthfeel) of the beer. Scores range from 2-5 points typically here, with 5 perfect for style, and lesser numbers for heavier, lighter body than style. 7. You have now evaluated the beer. Stop here and consider where the beer belongs in the scoring area at the beginning of this section, and decide on a "target" number. The exact number is often arrived at when considering what similar beers have scored in this judging flight, regardless of the actual number given a beer, you want to be sure if it scores better than previous beers, it will receive a higher score (and visa versa). Next subtract the scores you have already given in appearance and body to see what points you have left to give out. 8. Consider how you want to distribute the points relative to the comments you've written in each section, and record the aroma, flavor, and overall impression points. Use the overall impression area liberally to raise/lower the score to where it belongs, with a number from 1-9, which actually depends on the number of points given in the flavor/aroma sections. 9. Give feedback to the brewer. Make sure to give some positive comments, the appearance area, or at the beginning of the overall impression area are good places to write the comments. Give feedback to the brewer on how to eliminate probable causes of the flaws you've found in the beer. The Overall Impression area will likely be a good place to include this. 10. Check off any appropriate flavors you tasted in the Flavors Definition box of the scoresheet at the left hand side. This is a quick way to convey helpful information. -------------------------------------- From JudgeNet Wed Mar 4 21:13:08 1992 Subject: Re: Scoring Guide Reply to: RE>Scoring Guide >No real beer is a 50. >No beer deserves below a 19. Tom; I find it interesting with all the chat about the 20 vs 50 point debate that you have in effect created a 30 point system out of the AHA's 50 pointer. Given the range, shouldn't we take full advantage of it? I realize that there are not really any levels of un-drinkableness (?), however I have no problem giving a zero to an entry which gushes so badly that there isn't any left to sample. I agree that comments must be positive even when critical. Snide &/or smartass remarks have no place on a judging form. The credibility of the judge is at risk when such remarks are used in place of well reasoned thoughts. RW... -------------------------------------- From JudgeNet Thu Mar 5 00:13:05 1992 Subject: Scoring continued Russ, I agree with the effective limits that Tom Strasser's guide places on the 50 point scoring range. Even though keeping the range of beer scores between 19 and 49 restricts the range of possible points one can award a given beer, I believe that the res- pect accorded to *what* the points *mean* is enhanced enough to warrant this. At the low end, there is never a reason to distinguish an 18 from a 19, and so you may as well give a score that makes your point -- "your beer is undrinkable" -- but does not risk any form of anguish/humiliation/shame for the brewer. A 19 implies _optimism_ on the part of the judge...a bit more effort and you'll have a drinkable beer. And in reality, isn't that what it takes? If a brewer goes from being unclean to being clean, chances are good that his/her beer will go from being undrinkable to being drinkable. At the high end, we are only talking about one point, and unless the beer came out of the bright tank, it will have already deteriorated to the point where it'll be no more than a 49 anyhow. If the scoring range were from 0 to 100 and a 39 was the highest undrinkable score, I would still advocate giving the worst judgeable* beers a score in the range 30 to 39. With the range as is, I say that I see no reason to score a beer below 15, but that I personally have never scored a beer below 19. *For the example you cite, of a gusher where none remained in the bottle: I recommend that you *not* score it at all but rather offer extensive advice to the brewer and say "not scored". Same goes for a beer so obviously putrid (from just a whiff) as to be undrinkable. Brews to ya, STEVE -------------------------------------- From JudgeNet Thu Mar 5 11:15:47 1992 Subject: scoring guide Here are a few comments on the judging memo. Aroma and smell. Beer judges should take note of what wine judges do. That is, in order to really pick up aroma, you should inhale slowly through your nose from 4 to 6 times over a 30 to 60 second period. The aroma has to go way up in the nose behind your brow. This technique really lets aroma develop and reveals subtleties. Clarity. Really cloudy beers are a 0, most beers are a 1, and absolutely crystal brilliant beers are a 2. When I see a really "bright" beer, I write that down. I wonder whether the beer is filtered or not, and whether subsequent tasting might reveal that. A cloudy beer might be evidence of a poor boil, hasty racking, or something else. Head retention. Occasionally, you have no control over how the beer is being served, and it turns out the beer is served in glass that might have been washed with soap. So, you might try to notice whether the head disappears quickly across beers being served. Conditioning. How young or old in the bottle is the beer? Do you taste a hop-forward or young-hop flavor characteristic of recently brewed and bottled beer? Do you taste oxidation? Comments. Give written comments, be constructive, and have fun. If you enjoyed the beer, say "A very drinkable beer. Send me the recipe." or whatever. -------------------------------------- From JudgeNet Thu Mar 5 11:25:09 1992 Subject: Scoring beers less than 19? Well, I basically do what tom suggested, which is to not score a beer less than 19. I feel that the brewer will get the point and that there is no good reason to rub their face in it. On the other hand, what Russ says is true, the 50 point scale effectively becomes a 31 point scale by doing this. The problem, of course, is that little set of guidelines at the bottom of the score sheet: Excellent 40-50 An Excellent representation of style Very Good 30-39 A very good representation of style, has minor flaw(s) Good 25-29 A good representation of style, has significant flaw(s) Drinkable 20-24 A drinkable beer, but serious flaw(s) distinguish it from the style Problem 0-19 A beer with serious flaw(s) which prevent it from being drinkable, regardless of style. I'vew often wondered why Drinkable and Good beers only get a 4 point spread, but Problem beers get a 19 point spread. If I allowed to redesign the score sheet, one thing I would do is change each of the scoring categories to 10 points each, i.e. problem 0-10, drinkable 11-20, etc. to spread out the scoring of the beers more. As has been pointed out previously, most entries in competitions tend to get bunched in the high 20 - low 30 range, largely due to the way the scoring guidelines are set up. Steve Stroud PS - Tom, one comment on your scoring evaluation guidelines. You should make it clear that the way you have the points listed are only for the AHA score sheet. There are others in use - for example the HWBTA score sheet, which still assigns 10 points (out of 50) to appearance, or the one that Russ' club uses (20 point scale). The neophyte judge should be aware that they may run into these different forms on occasion. -------------------------------------- From JudgeNet Thu Mar 5 11:35:51 1992 Subject: Re: Scoring Guide >I find it interesting with all the chat about the 20 vs 50 point debate >that you have in effect created a 30 point system out of the AHA's 50 pointer. I don't think Tom has done this per se.. I think most judges feel that since the appearance points are virtually free, even a bad beer is gonna \score a minimum of 10, and there is a sort of a tacit prohibition on panning a beer, most folks (myself included) feel a beer must be undrinkable to score less than a 20, and if it does you MUST provide feedback as to what the problem is. I have given beers in the low teens, but there are always free points, so you really shouldn't ever give a beer a 0, though I have had beers that I felt did not deserve to be scored at all since they were obvious problems and felt it was a unfair to even attempt to score it, and instead just wanted to make comments. Of course since most of the competitions have computerized databases in the NE, I mwas basically forced to assign it a score, though I think for obvious problem beers it's meaningless. You must keep in mind the dual nature of the competitions. They are still intended to serve as a source of diagnostic expertise, and there are people who enter just to receive feedback they can't get elsewhere. Not everyone thinks their beer has any chance of winning, some folks just need some help. -JaH -------------------------------------- From JudgeNet Thu Mar 5 12:20:30 1992 Subject: Re: Scoring Guide >From Russ... } Tom; I find it interesting with all the chat about the 20 vs 50 point debate } that you have in effect created a 30 point system out of the AHA's 50 pointer. Just a couple of thoughts in addition to what Steve responded. First, while this has not been told to me explicitly, this seems to be the proactice of most experienced judges I had the pleasure of judging with, with master judge Rich Gleason being among these. In addition, I judged with Jay Hersch last weekend, and at the next table a first time judge wanted to give a beer a 5! Jay in no uncertain terms told the judge this was not warranted. Secondly, I think we should step back and take a look at the bigger picture. We're not getting paid to be judges, so what are we trying to do? My answer to this question is to promote the level of brewing for people whose beer I judge. I think this is best accomplished by not only giving constructive criticism about their beer, but also by making them feel as good as possible when reading the evaluation of their beer. This is why I suggested making sure positive comments were written, and it is why I am suggesting that beers not be scored below a 19. As Steve noted, there is no reason to differentiate whether a beer is a 17 or a 19, so why not encourage someone with a slightly higher score? Just my opinion. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Thu Mar 5 14:16:09 1992 Subject: Re: Scoring Guide It seems that the scoring system is broken for bad beers (and superb beers too?). The "19 points" problem is that a bad beer must be ranked less than 20 if it deemed "undrinkable" -- but the way I've seen scoresheets tallied is that everyone knocks points off of the subsections such as "appearance" just so the total equals 19. Would it make sense to allow negative points in the "overall impression" section so that a beer that is bad can be penalized rather than losing points in every section? For example, I made a Munich Dunkel this winter than had the right color, adequate clarity, good head -- but wowee did it reek of DMS (probably from bad yeast - Vierka dark lager dry, a yeast to avoid!). This beer was undrinkable. But if someone was entering such a beer, isn't it still worth providing information were possible? Bob Devine -------------------------------------- From JudgeNet Fri Mar 6 13:15:26 1992 Subject: Scoring, and Scoring Bad Beers >Reply-To: jdg00 at amail.amdahl.com At the January meeting of the Ann Arbor Brewer's Guild, we had an informal judging of pale ales for the AHA "Hail to Ale" competition. A group of about 5 of us used the 50 point system, and the rest of the crowd just tasted very informally. At the end, we voted for our favorite, which was sent on to Boulder. We had about 6 beers to judge. This was my first attempt at judging, and I tried very hard to be careful and objective. They were delivered by pitcher one at a time, and this made it especially difficult. I found I was losing sensitivity to aroma, taste, mouthfeel, and finish as the beers went by. My own beer came by in the 4th postition, and by that time I didn't even recognize it. Part of the problem was that one of the beers was terribly phenolic. So much so, that it hurt my ability to discern differences in the beers that followed it. I remember writing down notes that the phenolic overtones made it too difficult to discern malt/hops balance, or if there were other aromas and flavors to judge. I recall, though, that I scored it in the low teens. At the February meeting, I tasted the same beer again. NO phenolics anymore, and it was a good beer! I'd have scored it in the high 30s or lower 40s. The flaw was either isolated to the judging sample or it was taken care of by aging on top of yeast. To me, this means that flawed beers are the most difficult to judge. Perhaps a score of 19 is justified because one can't necessarily tell what could be good about the beer? Also, levels of perception differ. One of the 5 of us scored all 6 beers above a 40, and didn't perceive any flaw in the beer I had difficulty with. Also, he was the only one of us to be AHA/HBWTA Certified. I'll be taking my exam in May, and, if I pass, I may judge at the 2nd round of the nationals. In that case, perhaps I won't have any flawed beers to judge. :-) -------------------------------------- From JudgeNet Mon Mar 9 13:21:27 1992 Subject: Beer Scoring I've been corresponding with Steve Stroud as a side light to the tasting instructions from last week. We think we've come up with a more specific suggestion for scoring which allows a larger scoring range for the judges. As if you all hadn't heard it before, it's a 100 point system (what'd ya think Mike?)!, it would go along the lines of: } From: Steve Stroud } Actually, I like the idea of a 50 - 100 point scale, much like Robert Parker } uses in The Wine Spectator, with } } Excellent 90-100 An Excellent representation of style } Very Good 80-89 A very good representation of style, has } minor flaw(s) } Good 70-79 A good representation of style, has significant } flaw(s) } Drinkable 60-69 A drinkable beer, but serious flaw(s) } distinguish it from the style } Problem 50-59 A beer with serious flaw(s) which prevent it } from being drinkable, regardless of style. IMHO, I think this eliminates two flaws in the current judging system, and as Mike Fertsch has suggested, gives people a number they can relate to. The first problem it eliminates is it gives every beer at least a 50, which is psychologically more consoling to the competing brewer. I think this psychological reason is probably why in the 50 point system the definition of a problem beer is at 19 points, but this leaves the lower end of the scale practically useless (as previously discussed). On 50 point scale, however, this fact ends up "squishing" the drinkable range of beers to 5 points (where as all other ranges get 10 points). This leads to the second eliminated problem when using the 100 point system, the drinkable range can easily be expanded to 10 points, allowing equal flexibility as that which exists in the other ranges. Thus you can score a beer with serious style problems drinkable, and not have it lie just a couple of points above problem. I know I've felt on several occasions that beers I've judged were just a left 3 or 4 points from a problem beer, but the problem was stylistic, not drinkability. This should allow judges to make more usage of the drinkable range. Finally an added benefit, and why 100 points was chosen, is the education system tend to use a 100 point scale, so we all should be familiar with what a score means...I know if I didn't have judging experience I could relate to a 92 much better than a 42. Shouldn't a goal of the scoring system be that it is more readily understandable? Remember that this would really be a shifting of the current system, rather than an expanded scoring system, thus Russ won't have to worry about having too many possible number to assign :-). } This way, all of the ranges get ten points and a "Flawed" beer doesn't get }totally destroyed (i.e., a 59 out of 100 sounds a whole lot better than a 9 out } of 50!) Should we run it by JudgeNet and see who salutes? So... What do you think?????? Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Mon Mar 9 13:34:12 1992 Subject: Physiology of Tasting John DeCarlo sez... >Good job, Chuck, hope you enjoyed the exam on Sunday (or at least did really >well). I don't know anybody who enjoys the exam (its no carnival to adminster and score either). I think I got >90, it'll be a few weeks before I know for sure. >One thing that has struck me as I study for the BJCP exam is the apparent lack >of discussion about the act of tasting the beer itself. Rather it is the >beer that is studied. Shouldn't judges know about sensory levels, ppb >sensitivity, threshold levels of chemicals, taste areas of the tongue, >effect of temperature and carbonation levels on certain flavors, etc.? > >Am I missing something basic here? Should there be questions devoted solely >to *how* people taste beers? In effect, the beer judging that counts for 30% of the exam tests your ability to taste beer (in addition to your ability to communicate on a judging form). This is more useful than answering questions about tasting. More experienced judges will understand their own thresholds and perceptions well enough to provide a more objective description and analysis than a novice. The moral here is that it is useful to know about all the things you mentioned, but you should be prepared to demonstrate that knowledge instead of write about it. The best way to learn about your own perceptions is to attend a flavor perception demonstration like the Dr Beer sessions that Steve Stroud and Jay Hersch put on. I do plan on adding a discussion of the beer judging part of the exam to the study guide. ===== Chuck Cox SynchroSystems chuck at synchro.com -------------------------------------- From JudgeNet Mon Mar 9 22:15:30 1992 Subject: 100 point scale Hmmm, the more I think about it, the more I like the 0 - 100 point scale as opposed to a 50 - 100 point scale (the 50 - 100 point version is to hard to set up. Where do the first 50 points come from???) I'd do it like this, which is essentially a doubling of the AHA/HWBTA score sheet, with the appearance/flavor discrepancy averaged: aroma 20 appearance 15 flavor 35 body 10 overall 20 --- total 100 Ratings would be as Tom suggested, <60 major flaws 60-69 drinkable 70-79 good 80-89 very good 90-100 excellent I think that this is a scale that all homebrewers could easily relate to, and it would be no harder for the judges, since it's just a doubling of what we currently do anyway. In addition, it decompresses the current drinkable/good ratings, giving equal 10 point spreads. Steve Stroud -------------------------------------- From JudgeNet Mon Mar 9 22:25:12 1992 Subject: Re: Beer Scoring } From: Steve Stroud } Actually, I like the idea of a 50 - 100 point scale, much like Robert Parker >} uses in The Wine Spectator, with >} >} Excellent 90-100 An Excellent representation of style >} Very Good 80-89 A very good representation of style, has >} minor flaw(s) >} Good 70-79 A good representation of style, has significant >} flaw(s) >} Drinkable 60-69 A drinkable beer, but serious flaw(s) >} distinguish it from the style >} Problem 50-59 A beer with serious flaw(s) which prevent it >} from being drinkable, regardless of style. > >} This way, all of the ranges get ten points and a "Flawed" beer doesn't get >}totally destroyed (i.e., a 59 out of 100 sounds a whole lot better than a 9 out >} of 50!) Should we run it by JudgeNet and see who salutes? >So... What do you think?????? Excellent! I believe that this is a very good solution, for all of the reasons you stated. For discussion's sake, I made a succinct list of all of the advantages of the 100-pt scale proposed by Steve Stroud and Tom Strasser (with goading from numerous JudgeNet subscribers): 1. The 100-pt scale is familiar to the public (percentages, school test scores). 2. There is more "room to judge" in the 100-pt scale, i.e. less need for half-points, less ties. 3. The effective range of the clipped 100-pt scale is 50. This is better than clipping the 50-pt scale at 19 and leaving only a 31 point range or clipping the 20-pt scale and leaving a tiny effective range. 4. There is a ten point range for both "Drinkable" and "Problem" beers. a. More feedback to the brewer (instead of a 19 for every "Problem" beer). b. Equal sizes for the ranges (all are ten points). 5. Giving every beer at least a fifty is more psychologically satisfying to the brewer, while still telling him that his beer isn't up to snuff. Here are some disadvantages: 1. The 50-pt scale is being used widely. 2. Some judges are more comfortable with the 50-pt scale. Those that do not like the 100-pt scale are invited to add to this list of its disadvantages :) I am all for changing to the new 100-pt scale. One question, though: How do you make certain that all beers get at least a fifty? Do you tell the judges that they have to score beers at least a fifty and then make all the scores start at zero (my own preference)? Or do you make all the scores start at fifty and go up? Mike Hall mlh at cygnus.ta52.lanl.gov -------------------------------------- From JudgeNet Wed Mar 11 09:18:42 1992 Subject: Beer Scoring I really like the 100 point system. I feel that it would improve the feedback to the brewer for all the reasons that have already been discussed. If there is widespread support for the idea, the next step would be to try it out in an actual competition. Any volunteers? ===== Chuck Cox SynchroSystems chuck at synchro.com -------------------------------------- From JudgeNet Wed Mar 11 10:22:49 1992 Subject: Re: Beer Scoring -------- Regarding the 100-pt scale, Mike Hall lists several of the advantages: ... > 2. There is more "room to judge" in the 100-pt scale, i.e. less > need for half-points, less ties. I'm not sure that this is necessarily a good thing. One advantage of the 50 point scale is that it tends to lump similar beers very close numerically, forcing the judges to go back and take another taste and reconsider them. Usually the judging team then decides which should go on and the scores may (or may not) be adjusted accordingly. The current system is already highly subjective and a larger point spread may worsen that. Here are some questions about the 100-pt idea that I have: - How close (numerically) do all judges need to score the same beer in a round? Under the 50 pt. system, we usually tried to keep all scores within 7 pts. - If two entries are within, say 5 pts (or some number of points), should they be re-tasted? Does that then imply that we should probably round scores to the nearer 5 pts except in cases that are close? I like the idea of the 100-pt system, but it might make it harder to be consistent from beer to beer and judge to judge. Remember that the score number is really only a number. The true feedback on the score sheet is (or should be) in the written comments that the judge writes to the brewer. > Mike Hall > mlh at cygnus.ta52.lanl.gov Glenn Colon-Bonet gcb at fc.hp.com -------------------------------------- From JudgeNet Wed Mar 11 10:32:57 1992 Subject: Re: Beer Scoring Glenn Colon-Bonet expresses some concern about the effect of the 100 point system on relative scores. To oversimplify a bit, the 100 point system is just a 50 point 'offset' added to the current 50 point system. So for the most part, relative scores would remain the same. For instance two beers scored as 35 & 40 in the current system would probably be scored around 85 & 90 in a 100 point system. As others have already explained, the 100 point system does allow for some restructuring of the excellent/drinkable/problems scale, but I don't think that would have much effect on relative scores. ===== Chuck Cox SynchroSystems chuck at synchro.com -------------------------------------- From JudgeNet Wed Mar 11 11:25:25 1992 Subject: Re: Beer Scoring } From Chuck Cox: } I really like the 100 point system. I feel that it would improve the feedback } to the brewer for all the reasons that have already been discussed. If there } is widespread support for the idea, the next step would be to try it out in an } actual competition. Any volunteers? The esteemed Dwight Beebe is on this list and is running a competition this weekend here in Ithaca! Not only that, one Pat Baker is supposed to be coming, as well as a guy some people refer to as Michael Jackson. Maybe he might be convinced to take a walk on the wild side with this new scoring system :-) Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Wed Mar 11 12:30:51 1992 Subject: Re: Beer Scoring thought i had posted this before, but... while i think the 50 point system is ok i can see merits to a 100 point system since it is natural to most evryone and expands the range in which good beers can lie, thus easing potential ties... equally intuitive IMHO is a plus/minus system. thism rewards characteristics appropriate for style, detracts for off flavors or inappropriateness for style. in such a system beers would rate as follows -25 to -15 problem -15 to -5 drinkable -5 to +5 good +5 to +15 very good +15 to +25 excellent a proposed breakout of sections is basically a modification of a 50 point scale... i have deferred breaking down the point allocation within the sections for the time being. AROMA total +/- 5 malt, hops, balance as appropriate for style presence/absence of off aromas APPEARANCE total +/- 3 color, clarity, head FLAVOR total +/- 11 malt, hops, balance as appropriate for style condition aftertaste BODY total +/- 1 full or thin, or in-between OVERALL IMPRESSION total +/- 5 Now I have not scored with this system, but i think the increase of weighting to flavor away from body is justified. Body while important is either there or it isn't so i don;'t think it merits a 5 point spread. I think that flavor is what most people drink a beer for, a nd thus it is the single most important component. The basic reason i like this system is it credits a beer for it's good points, and penalizes for the bad. I don't think that the present 50 point system (nor a 100 point) does that. my impression of how most folks judge a beer is to always award points, with the award being more or less dependent on the beers characteristics. i would rather penalize the bad and award the good. yet another $0.02 -JaH -------------------------------------- From JudgeNet Wed Mar 11 12:33:30 1992 Subject: Re: Re- Beer Scoring Reply to: RE>Re: Beer Scoring >As others have already explained, the 100 point >system does allow for some restructuring of the >excellent/drinkable/problems scale, but I don't think >that would have much effect on relative scores. And it certainly won't cause problems in the selection of the winners' scores since they don't come out of the drinkable/problems range. RW... -------------------------------------- From JudgeNet Wed Mar 11 14:16:44 1992 Subject: Volunteer (maybe) Greetings! Thanks, Tom, for the compliment, but I know you're just looking for an easy mark. I've been following the thread on the proposed expansion of the point scale with interest. I believe it is worth the effort to try it out and I'm willing to give it a whirl. My only hesitation is in not wanting to spring a new judging system on the fine group of people who have volunteered their services. I will contact the judges tonight and see how they feel, if there is no strong objection I'll alter the scoreing forms and use them on Saturday. I'll make an effort to sound out the judges after the competition and will report their comments/criticisms. BTW, kudos to Chuck for the fine job he's done on the study guide, I wish I'd had something like it prior to my exam. Cheers! Dwight (the esteemed) Beebe P.S. Remember that there's still time to get all those great porters and stouts to our contest if you'd like. Forms are available on-line and second day air will ensure your beers make it by Friday. Contact me or our Judge Director, the esteemed Dr. John Lenz at jelj at cornella.cit.cornell.edu for the gritty details. -------------------------------------- From JudgeNet Wed Mar 11 15:23:35 1992 Subject: negative feedback I disagree with Jay's suggestion of going to a +/-25 point scale. I think it's important to give positive feedback to brewers even if their beer is not too good (this year). We all want homebrewing to flourish and we should encourage all to join our ranks and keep trying to make better beer. If I had gotten a -20 on my first competition entry, I may have quit brewing. I like the 100 point scale, proposed earlier and even think the arguments against the 100 point scale were overrated. Al. -------------------------------------- From JudgeNet Wed Mar 11 19:17:24 1992 Subject: Re: The scoring debate I've been watching this thread with great interest and feel like it is now time to add my $.02 worth. I am not a judge, but I have been studying for a while, mostly in an effort to better educate myself about beer styles and defects. I figure that the knowledge will help make me a better brewer. One of these days I'll take the exam when "real-life" stops interfering. For what it's worth, I really like the idea about the 100 point system and have often wondered why it wasn't in use already. Back to lurking... -- Kevin -------------------------------------- From JudgeNet Wed Mar 11 19:27:51 1992 Subject: Update on study guide; +/- scoring forms Study Guide stuff: The name of the grape (muscat) lambic brewed by Cantillion is "vigerrone" as in kriek = cherry, vigerrone = grape. You might want to update the study guide. Also, some of the info about judgin points is incomplete. Here is some updated info (taken from the current BJCP by-laws). BEER JUDGE CERTIFICATION PROGRAM You should know how the BJCP is organized and what the requirements are for the various ranks. A more complete description of this information is included in the BJCP By-Laws, available from the BJCP Administrator, 736 Pearl Street, PO Box 1679, Boulder, CO 80302-1679. American Homebrewers Association - AHA Home Wine and Beer Trade Association - HWBTA Beer Judge Certification Program - BJCP Ranks: exam score / experience points (50%+ must be from judging) Recognized 60+ / 0 Certified 70+ / 5 National 80+ / 20 Master 90+ / 40 Honorary Master Temporary designation Experience Points: small / large / national Organizer 2-10, depending on # of entries Asst Organizer up to 5, by organizer from his points Best of Show 1 / 2 / 5-6 Judge 0.5 / 1 / 2-3 Steward 0 / 0.5 / 1 Exam Administration 0-2 Sanctioned Competitions Small Regional (<75 entries) Large Regional (75+ entries) National - annual AHA & HWBTA competitions Judging Form & Scoring ********************** Scoring form: There have been a few comments here recently about +/- scores on a judging form. A form similar to the one Jay described was in use here in New England when I first started brewing a decade ago. It was quickly abandoned. I believe that a serious problem with such a system is it's ability to give a truly bad beer a negative number. If we're so worried about giving a faulty beer anything <19, imagine the brewhaha that would break out if judges started assigning negative totals to beers. This is a hobby to encourage people and I think that the +/- scoring system has the ability to do the opposite. Not a good idea, IMO. Steve Stroud -------------------------------------- From JudgeNet Wed Mar 11 19:27:51 1992 Subject: Update on study guide; +/- scoring forms Study Guide stuff: The name of the grape (muscat) lambic brewed by Cantillion is "vigerrone" as in kriek = cherry, vigerrone = grape. You might want to update the study guide. Also, some of the info about judgin points is incomplete. Here is some updated info (taken from the current BJCP by-laws). BEER JUDGE CERTIFICATION PROGRAM You should know how the BJCP is organized and what the requirements are for the various ranks. A more complete description of this information is included in the BJCP By-Laws, available from the BJCP Administrator, 736 Pearl Street, PO Box 1679, Boulder, CO 80302-1679. American Homebrewers Association - AHA Home Wine and Beer Trade Association - HWBTA Beer Judge Certification Program - BJCP Ranks: exam score / experience points (50%+ must be from judging) Recognized 60+ / 0 Certified 70+ / 5 National 80+ / 20 Master 90+ / 40 Honorary Master Temporary designation Experience Points: small / large / national Organizer 2-10, depending on # of entries Asst Organizer up to 5, by organizer from his points Best of Show 1 / 2 / 5-6 Judge 0.5 / 1 / 2-3 Steward 0 / 0.5 / 1 Exam Administration 0-2 Sanctioned Competitions Small Regional (<75 entries) Large Regional (75+ entries) National - annual AHA & HWBTA competitions Judging Form & Scoring ********************** Scoring form: There have been a few comments here recently about +/- scores on a judging form. A form similar to the one Jay described was in use here in New England when I first started brewing a decade ago. It was quickly abandoned. I believe that a serious problem with such a system is it's ability to give a truly bad beer a negative number. If we're so worried about giving a faulty beer anything <19, imagine the brewhaha that would break out if judges started assigning negative totals to beers. This is a hobby to encourage people and I think that the +/- scoring system has the ability to do the opposite. Not a good idea, IMO. Steve Stroud -------------------------------------- From JudgeNet Wed Mar 11 19:43:57 1992 Subject: Re: negative feedback >If I had gotten a -20 on my first competition entry, I may have quit brewing. getting a -20 is the equivalent of getting say a 10 under the present system. i think it is fallacious to just look at the raw number, under the proposed 100 point system the -20 is a 55, how happy would you getting a 55 test score. of course the most important reason this is a bad argument is that a beer receiving this type of score in any system is heavily flawed and the score is essentially pointless, a beer this bad must be appraised by the person receiving this score for the comments it receives. i think the people who submit a beer this poor that don't know they have serious problem and actually think the beer has a chance (barring that it is isolated to only the specific bottle judged) are few and far between and have serious misconceptions about beer styles and flavors..., so the real comparison here is the present systems 20-50 range, the 100 pt systems 60-100 range, and the +/- systems -15 to +25 (i.e. the range for drinkable beers to excellent beers). mthe +/- system has the same approach of using a wider range (40 points) for drinkable to excellent, but IMHO it doesn't take the approach that the first X pts are free... in the 50 and 100 pt systems you essentially get 20 or 50 "free" pts respectively due to the way the scales are set. with the +/- system rather than thinking "this beer has a phenolic aroma, should i give it 5 pts or 6??" you would say, "this beer has a phenolic aroma, i'll detract 2". i think it is much more natural for a judge, to identify flaws and detract points, and identify true characteristics of style and award them... as an experiment it would be interesting to gather a group of judges and give them a panel of say 15 beers, and ask them to use the 50 pt on 5, the 100 pt on 5, and the +/- on 5, of course to be useful the 5 in each should be the same, but to keep the judges honest really only 2 or 3 of the 5 used on each form would be the same across all forms. hopefully this might show some indications of how judges would use the forms, and which they like (it would help to work with judges not participating in this discussion to help reduce bias). -jay -------------------------------------- From JudgeNet Wed Mar 11 21:14:07 1992 Subject: minus scoring system I think that from a psychological standpoint, having negative numbers can be percieved incorrectly. We are conditioned by the current school system grade scale to have certain feelings about numeric scores, and getting a zero as a score does not feel right. The rating might be perfectly fair, but the *perception* of it must be correct for the right impact (as Algis mentioned, to enhance the art of brewing), I feel very strongly about this. I was also put off by the comments of a first round judge who rated a brew that I had tried and found to be quite good. His words were "This just don't do it for me", with no explanation of character/style defects. When I directed the first Wines Inc. competition I made a point of asking for comments that could be interpreted constructively. I feel it makes a big difference to the particpants. -- John Isenhour, The Hop Devil renaissance scientist and AHA/HWBTA certified Beer Judge hopduvel!brewmaster at linac.fnal.gov isenhour at lambic.fnal.gov -------------------------------------- From JudgeNet Thu Mar 12 03:14:37 1992 Subject: negative scores There are two issues about giving negative scores: 1. negative scores as a overall value (as in "my beer got a -23!") 2. negative scores in a section (as in "this very yucky looking beer gets a -3 points for appearance") I think nobody really likes to give a negative score because it is seen as too heavy a rejection. However I like the idea of positive/negative ranges for each section: appearance [-6..+6] body [-3..+3] (etc) Negative points make it clear that something is wrong. Put yourself in the shoes of a beginning brewer who got, say, a `3' for appearance. Is that good? Is that bad? Or was the judge just giving every beer a 3? While the written comments should make it clear, I've seen score sheets that didn't supply enough comments to help. A base score of, say, 50 could be used as the starting number and then have added or subtracted from that to yield the final score in 0..100 points. Bob Devine -------------------------------------- From JudgeNet Thu Mar 12 09:16:14 1992 Subject: +/- scoring system I want to restate my opposition to the +/- scoring system, for several reasons: Jay sez: }Getting a -20 is the equivalent of getting say a 10 under the present system. This is true, but virtually no one gets a 10 in today's competitions, so we can assume that -20's will also be very rare. More importantly, however, are the drinkable/good ranges. You set up the +/- so: -25 to -15 problem -15 to -5 drinkable -5 to +5 good +5 to +15 very good +15 to +25 excellent So comparing the four proposed scoring systems, we get the following scores for a middle scoring "good" beer: +/-: 0 20 pt.scale: 12 (?) 50 pt. scale: 28 100 pt. scale: 75 I don't know about the rest of you, but if I entered a "good" beer in a competition and received a zero score, I wouldn't be smiling about it. Would you? Think of the psychological impact on the brewer! Because of this, I think that most judges would avoid the negative range except for the absolute worst beers, and we'd be right back in a "compressed" 5-25 point scale for most entries. Avoiding the compression was the thing that started off this discussion in the first place. Comparing the other three scales above, the "75" that a good beer receives just looks the most natural to me. It's the kind of number that we all grew up with in our educational system and relate to innately. After talking myself around this issue, I think that the 0-100 scale makes a lot more sense and is easier to deal with than the 50-100 scale. It also gives the judges more points to play around with; in addition, it reduces the importance of the appearance points: aroma 20 appearance 15 flavor 35 body 10 overall 20 total 100 In other words, a beer could have a perfect appearance and it would only have a small impact on the final score, something that is a problem with some of the current score sheets. It has been implied in previous postings that there is no requirement for AHA/HWBTA sanctioned competitions to use any kind of standardized scoring form. I hope that someone (Hi Dwight!) will consider using the 100 point scale form in an upcoming competition. Steve Stroud -------------------------------------- From JudgeNet Thu Mar 12 10:18:48 1992 Subject: Points On the topic of the relative merits of the 50-point vs. 100-point system, it seems to me that the system could stand work in more productive places than the total number of points: - As mentioned earlier, does body really need such a spread? - Is aroma really as important as the points allocated? Many styles have no major aroma component; while an IPA should give one the sense of being in Kent, or a pilsner the sense of being on a Czech Saaz hop farm watching scantily clad high school girls pluck blossoms, how much aroma is necessary for Doppelbock? Or Weiss? - Isn't drinkability what it's all about? I personally would favor an additional scoring component that would allow beers that are excellent but not true to style in some way to get great positive feedback, but no be eligible to win. Many beers I have judged end up tasting great (but not less filling), but something is not quite right - a roasted hint in porter, too deep a color for pilsner, too much body for northern brown ale, etc. These are real "winners" in the sense of a brewer's success, but not "the ultimate scottish ale". I know when I first started I had no idea what category to enter a brew in - and the situation's getting worse with the 3,227 categories in this year's nationals. Be serious - who can tell whether an IPA has British or American hops? Who cares? We should have some way of letting the brewer know he (and they almost always are a he) did a great job, point out the out-of-style element, but still give a nice score. Pardon the ramble. Eric Webster, CAMRBIMF (CAMpaign for Real Beer In My Fridge) -------------------------------------- From JudgeNet Thu Mar 12 10:34:04 1992 Subject: BJCP BEER JUDGE CERTIFICATION PROGRAM You should know how the BJCP is organized and what the requirements are for the various ranks. A more complete description of this information is included in the BJCP By-Laws, available from the BJCP Administrator, 736 Pearl Street, PO Box 1679, Boulder, CO 80302-1679. American Homebrewers Association - AHA Home Wine and Beer Trade Association - HWBTA Beer Judge Certification Program - BJCP From Steve Stroud > Asst Organizer up to 5, by organizer from his points This should be Asst Organizer up to 5 Assistant Organizer points are now independent of the Organizer points. The Organizer may also divide up their own points to assistants, if they wish. This proved unpopular, so assistant org points were created. Jim Homer BJCP Co-Director att!drutx!homer -------------------------------------- From JudgeNet Thu Mar 12 10:34:04 1992 Subject: BJCP BEER JUDGE CERTIFICATION PROGRAM You should know how the BJCP is organized and what the requirements are for the various ranks. A more complete description of this information is included in the BJCP By-Laws, available from the BJCP Administrator, 736 Pearl Street, PO Box 1679, Boulder, CO 80302-1679. American Homebrewers Association - AHA Home Wine and Beer Trade Association - HWBTA Beer Judge Certification Program - BJCP From Steve Stroud > Asst Organizer up to 5, by organizer from his points This should be Asst Organizer up to 5 Assistant Organizer points are now independent of the Organizer points. The Organizer may also divide up their own points to assistants, if they wish. This proved unpopular, so assistant org points were created. Jim Homer BJCP Co-Director att!drutx!homer -------------------------------------- From JudgeNet Thu Mar 12 11:48:29 1992 Subject: +/- I sent a copy of my last posting to Charlie Papazian on compuserve. He responded: ************ Hi Steve. Thanks for the note. I was unaware that there was any conversation going on about scorsystems. Thanks for bringing me up to adate. Interesting about the +- discussion. You know that the AHA used to use a +- system for a couple of years . But after further onsideration, and mostly for the reason you suggested, as well as other persuasive constructive criticsm, we abandoned it. The 100 point system may have a bit more merit these days as the shape of the National changes. We'll discuss it here. Thanks, Charlie ************ Steve PS-Chuck, This brings me around to another more general question about who is and who isn't subscribed to JudgeNet. There are several national figures who have access to, but have never subscribed to JudgeNet. Charlie Papazian, George Fix, and Ed Busch (AHA BOA) all come to mind. There are probably others as well. Do you have to get a formal request from them to subscribe, or would you consider going ahead and adding someone like Charlie? Just curious. -------------------------------------- From JudgeNet Thu Mar 12 12:05:13 1992 Subject: Re: negative feedback >>If I had gotten a -20 on my first >competition entry, I may have quit brewing. > >getting a -20 is the equivalent of getting say a 10 >under the present system. > >i think it is fallacious to just look at the raw number, >under the proposed 100 point system mthe -20 is a 55, >how happy would you getting a 55 test score. I think you're missing my point. It's all psychological. I'm no psychologist, but it doesn't take one to know that punishment is usually counterproductive. I feel that the +/- scale you've proposed Jay, punishes the brewer more than a 100 point scale (I would be much happier getting a 55 than a -20). > >of course the most important reason this is a bad argument is >that a beer receiving this type of score in any system is heavily flawed >and the score is essentially pointless, a beer this bad must be >appraised by the person receiving this score for the comments it >receives. i think the people who submit a beerv this poor that >don't know they have serious problem and actually think the beer >has a chance (barring that it is isolated to only the specific >bottle judged) are few vand far between and have serious misconceptions >about beer styles and flavors..., No one was born knowing about beer styles and flavors -- we all had to learn -- and I'm *still* learning (it's one of the reasons I became a judge: to get more opportunity to learn). > >so the real comparison here is the present systems 20-50 range, >the 100 pt systems 60-100 range, and the +/- systems -15 to +25 >(i.e. the range for drinkable beers to excellent beers). Does a score of "0" sound like your beer was drinkable? > >mthe +/- system has the same approach of using a wider range >(40 points) for drinkable to excellent, but IMHO it doesn't >take the approach that the first X pts are free... in the 50 and 100 >pt systems you essentially get 20 or 50 "free" pts respectively >due to the way the scales are set. > So what? Give away free points -- who cares? The bottom line is that we want the judging to be: 1. fair, 2. helpful, and 3. (at least I feel) encouraging >with the +/- system rather than thinking "this beer has a phenolic aroma, >should i mgive it 5 pts or 6??" you woulkd say, "this beer has a phenolic >aroma, i'll detract 2". i think it is much more natural for a judge, to >identify flaws and detract points, and identify true characteristics of >style and award them... > I agree your system would be marginally easier for judges to use, but let's hope that no judge at the regionals or national is so stupid or inflexible to be able to adapt to a new system. I feel that each beer should be judged starting with a clean slate. Starting from there, you just go down the judging sheet and make your assessment, one area at a time. Right? The sheet tells you the range of acceptable answers. > >as an experiment it woulkd be interesting to gather a greoup of judges >and give them a panel of say 15 beers, and ask them to use the 50 pt >on 5, the 100 pt on 5, and the +/- on 5, of course to be useful the 5 >in each should be the same, but to keep the judges honest really only >2 or 3 of the 5 used on each form would be the same across all forms. >hopefully this might show some indications ofm how judges would use >the forms, and which they like (it would help to work with judges >not participating in this discussion to help reduce bias). > I agree this would be a good way to determine how easy the proposed system would be for judges to use. It would be more difficult to develop a way to assess the different types of scores on the entrant brewers. > > -jay > > I got into homebrewing because it was fun and because I love beer. I became a judge so I could help others be better brewers and to improve my own brewing. I think we owe our hobby a lot -- we've all gotten a lot out of it -- and we should be very mindful of the feelings of beginners. I vote for the 100 point scale because it increases the accuracy of scoring, while at the same time providing a larger number of "free points." Who does that hurt? Al. -------------------------------------- From JudgeNet Thu Mar 12 13:18:12 1992 Subject: Re: +/- scoring system Pardon me for being a late comer in the discussion. Is the compression an issue because the overall quality of beer has improved over the last few years? Or is it an issue because it makes it a pain to numerically determine which is the "best" beer in a judging? In the few club-only tastings I have become pretty comfortable with the current point system. I know that a 20 is pretty bad and a 40 is pretty good. Why would doubling the numbers make any difference? Or is this discussion driven by the 6 points allowed for appearence and how it affects that 30pt range from ok to great beers? - Larry Barello -------------------------------------- From JudgeNet Thu Mar 12 13:32:09 1992 Subject: new subscribers Steve Stroud sez... >There are several national figures who have >access to, but have never subscribed to JudgeNet. Charlie Papazian, George >Fix, and Ed Busch (AHA BOA) all come to mind. There are probably others as >well. Do you have to get a formal request from them to subscribe, or would you >consider going ahead and adding someone like Charlie? Just curious. I have not solicited subscribers, other than to post information about this list to the HBD. If you think there is an individual on the net who should know about this list, I encourage you to tell them about it. I would like to add all the folks you mentioned to the list, but I don't have the time or inclination to recruit them. I don't think it would be appropriate to add someone to the list without their permission. Somebody has to pay for the email, and they might not be amused to suddenly start getting unsolicited messages. Here is my standard informative blurb about the list, please distribute freely: (Perhaps someone should post this to the Compu$erve forum) ========================================================================== The Beer Judge mailing list is an Internet mailing list dedicated to the discussion of issues of interest to beer judges and competition organizers. Anyone with an interest in judging or organizing beer competitions is welcome to join. To subscribe to the mailing list, send your email address, name, and BJCP rank (use 'apprentice' if not ranked) to: judge-request at synchro.com ========================================================================== Chuck Cox SynchroSystems chuck at synchro.com -------------------------------------- From JudgeNet Thu Mar 12 13:32:30 1992 Subject: YALO (Yet another lurker opines) Just a few opinions/observations: 1. I think most people who have tried to make beer themselves are surprised to discover that it's almost as difficult to make truly horrible beer as it is to make truly great beer. Most fall somewhere in between (under the hump of the old bell curve). This is why I have no problem with the 0-100 pt. scale with a bottom limit of 50+ pts. Just like in grade school, a 50 means you didn't pass - try again. Unless a beer is truly toxic, I don't think it deserves a 0 score, or anything close to 0. If the beer is so obviously putrid that the judges can't (or won't) go near it, don't grade it at all. 2. This is also why I don't like the idea of a +/- system. Once you get into negative numbers, you start having to describe "How BAD was it?" Imagine if you will a Homebrew variant of Monty Python's "Australian Wine" sketch. While this might be good for a few laughs, I don't think it would offer too much in the way of instruction or encouragement to the person whose beer is being judged. IMO, it is sufficient to say "Your beer didn't pass and this is why..." 3. As for the the statement that setting the "floor" at 50 pts means that the first 50 pts are "free"...well, sure! Why not? Doesn't everyone like free stuff? Life is hard - beer should be fun. Just because someone submits "bad" beer, there's no reason to rub their nose in it. (Bad brewer! Bad brewer! ;-) 4. Also...has anyone else noticed that the Beer Judges themselves are graded on a 0-100 pt scale? A score of < 60 means - you guessed it: "You didn't pass - try again." -Rich Lenihan (Apprentice) -------------------------------------- From JudgeNet Thu Mar 12 15:15:02 1992 Subject: Reply to "+/- scoring system" (03/12/92 11:18:16) Hmmm. These discussions have raised in my mind the various approaches to giving points. Many of us are familiar with math tests where you start with 100 points and lose points for things you did wrong. When presented with the juding forms and something like 10 points for aroma, my thought has always been to wonder whether it makes any difference whether I subtract 2 because of a defect and get an 8, or whether I say that the aroma gets a C for average and therefore a 7, or whether I should be building up and saying it gets 6 points for these good things. The third approach leads to lower scores when I try it at home (I know, don't try this at home, kids, we're professionals). The second seems the easiest, and the third seems more like what I have been used to in grading situations. (Imagine how hard it would be if you had to find 87 good points on a math test instead of 13 points in mistakes.) Is there a de facto standard here? I sense from people's reactions to the different point systems that there are all three of these approaches being used. Internet: jdecarlo at mitre.org (or John.DeCarlo at f131.n109.z1.fidonet.org) Fidonet: 1:109/131 -------------------------------------- From JudgeNet Thu Mar 12 15:27:03 1992 Subject: Judging Approach Hi, I am new to the list, and have been following a bit of the discussion of various scales. I think I might have a bit of insight that may be helpful. Credit for this approach goes to Dr Michael Lewis of UC-Davis. He said that, when tasting a beer, one should approach it with the assumption that it is going to be a totally average brew, not the best you'll ever have tasted or the worst. Then, as you actually taste it with each sensory component and the beer style in mind, you should increment or decrement your judgement away from "average." Note that this sort of approach works just as well with a +- scale as with a 100 point scale. Dr Lewis made these comments at last summer's microbrewery class. He also offers a whole two-day class on the sensory evaluation of beer, but I didn't attend that. -------------------------------------- From JudgeNet Thu Mar 12 16:16:08 1992 Subject: Re: negative scores and how many of you were upset at getting a 3.5 in college after getting test scores back in the 0-100 range all your lives?? did you really think it meant you got a 3 1/2.. no you understood what your score meant because you knew the range in which it was given. the same holds here. i think this is a really stupid argument. it really doesn't matter what the score for a good beer is. people are intelligent enough to know that on a +/- scale a 0 is a median score. so in a 100 pt scale if people view a 60 as a failing grade then all you really have is a 0-40 pt range. i think you give very little credit to people's intelligence by assuming that they judge the score they receive independent of the scoring system. the real critical factors to look at are the range for problem to excellent. the +/- system and the 100 pt are identical in this respect. the second critical factor is how points are allocated into each section, +/- 100 50 (present) 50 (old) aroma 5 (20%) 20 (20%) 10 (20%) 10 (20%) appearance 3 (12%) 15 (15%) 6 (12%) 10 (20%) flavor 11 (44%) 35 (35%) 19 (38%) 15 (30%) body 1 (4%) 10 (10%) 5 (10%) 5 (10%) overall 5 (20%) 20 (20%) 10 (20%) 10 (20%) problem 10pts 60 20 20 drinkable 10 10 5 5 good 10 10 5 5 very good 10 10 10 10 excellent 10 10 10 10 of course the single most important point behind the +/- system was addressed very well by Bob Devine, and that is my original assertion that it allows the judge to reward positive components, and penalize negative ones. he correctly points out that it is more understandable for an entrant to know that a -3 for aroma is bad, as opposed to getting a 3. Since many of you feel strongly about negative scores, Bob's suggestion of readjusting the +/- system into a 100 point scale with 50 as the mean seems like a good compromise. I don't see anything wrong with a 0 centered system myself, I would be quite interested in working with Bob and others to create a hybrid system which provides points in a positive range, yet still lets the judge deduct for flaws and reward good points the way the +/- system does. -JaH -------------------------------------- From JudgeNet Thu Mar 12 16:43:54 1992 Subject: One more lurker comes forth! Well, I've been watching this discussion with great interest. Personally, I prefer the 100 point system for many of the reasons stated by others, especially because of the argument that it is "natural" to us in the sense that we are all so very familiar with a 100 point scale in a variety of contexts. Now, on to the point: It seems that most of the ground has been covered on this topic as there is beginning to be quite a bit of redundancy in the various arguments. I propose that we take a vote and encourage any other "lurkers" out there to participate. I for one would like to see what the final counts would be. If the outcome indicates a consensus decision in favor of a new scoring system, we might be able to start the wheels of change turning. After all, we have the ear/terminal/workstation of one of the co-directors of the BJCP (hi Jim!). I'm not sure how many people are on the list, but I'd bet it is a sizeable percentage of those enrolled in the BJCP and would likely carry some amount of clout. Dave Resch -------------------------------------- From JudgeNet Fri Mar 13 12:17:31 1992 Subject: Low Score Anecdote With all this talk of changing the scoring system, and how to judge bad beers, I though I'd relate a little story. At the BJCP class I'm taking, I noticed that one of the other students had a laminated scoresheet in his notebook. I asked him if he laminated it to use with grease pencil for practice, and he said, "No, this is the results from one of the beers I submitted to the Nationals." So, I took a look at it. It was a review of a 22 point chlorphenolic/phenolic beer. The judge had found many nice things to say about it...color, malt/hop balance... but most importantly, it said, "Please see back for more information." On the back was a list of probable causes and the comment, "If the flaws can be eliminated through pre-boiling your brewing water or more careful attention to sanitation, this beer could be an award winner." He saved the judge's report, pre-boiled his brewing water, and took a bronze the very next year. Now, to me, this is what judging of flawed beers is for: to help people find out how to improve their beers, in as positive and construtive a manner as possible. -Josh Grosse, Apprentice- -------------------------------------- From JudgeNet Fri Mar 13 17:15:33 1992 Subject: Scoring systems, again (are we tired of this yet?) I really don't want to beat an dead horse, and I agree that a lot of the arguments over the 20 vs 50 vs 100 vs +/- point systems are beginning to get repetitive. The majority of responses appear to favor some sort of 100 point scale, just to even out the drinkable/good/v. good/excellent ranges if nothing else. However, I would like to respond to one thing that Jay put in his last posting. I was looking at the chart that he made up demonstrating the %'s that each characteristic makes up from the total score: +/- 100 50 (present) 50 (old) aroma 5 (20%) 20 (20%) 10 (20%) 10 (20%) appearance 3 (12%) 15 (15%) 6 (12%) 10 (20%) flavor 11 (44%) 35 (35%) 19 (38%) 15 (30%) body 1 (4%) 10 (10%) 5 (10%) 5 (10%) overall 5 (20%) 20 (20%) 10 (20%) 10 (20%) While I can't disagree with the numbers, it is another example of statistics that can be deceiving. I don't believe that it is the % that each characteristic adds to the TOTAL score that is important, it is the % of how far it gets the score to the problem/drinkable line. For example, no one really cared that in the old 50 point scale appearance was 20% of the total score. What everybody griped about was that a perfect looking beer could be mediocre/poor in other characteristics and would still get a "drinkable" or "good" final score because the "appearance" already was half way to drinkable (10 points appear/20 points drinkable). It was hard to keep the point total down in some cases. If we go back and look at Jay's chart, but the percentages are calculated as: (Points from characteristic) / (points needed to reach drinkable level) we get: +/- 100 50 (present) 50 (old) aroma 5 (50%) 20 (33%) 10 (50%) 10 (50%) appearance 3 (33%) 15 (25%) 6 (30%) 10 (50%) flavor 11 (110%) 35 (58%) 19 (95%) 15 (75%) body 1 (10%) 10 (17%) 5 (25%) 5 (25%) overall 5 (50%) 20 (33%) 10 (50%) 10 (50%) where problem 10pts 60 20 20 drinkable 10 10 5 5 good 10 10 5 5 very good 10 10 10 10 excellent 10 10 10 10 These numbers are very interesting and point out a benefit of the 0-100 point system that had never occurred to me before. Namely, the 60 points needed to get to a drinkable beer aren't "free", they are a buffer to keep any single characteristic of a beer from skewing the final rating of the beer. Because of this buffer, appearance has the least effect (of the four scales) in the 0-100 scale (note that this wouldn't be true in a 50 - 100 scale), since it is only 1/4 of the points needed to reach drinkable. This observation has also made me rethink some of our earlier discussions. I think that I better understand why the current 50 pt scale starts drinkable at 20 instead of 10. Imagine how hard it would be to keep a good looking but seriously taste-flawed beer in the problem range (<10) when it already had 5 or 6 points for appearance. A judge would have to give a zero to almost every other characteristic on the score sheet. To a lesser extent, the +/- system would also suffer from the same problem, since it, too has no buffer built into it and a judge would have to rate every other characteristic of a perfect looking beer as very negative just to keep it in the drinkable range. Cheers, Steve Stroud -------------------------------------- From JudgeNet Fri Mar 13 17:29:17 1992 Subject: Re: Re- negative scores Reply to: RE>Re: negative scores The system we use at the Malts' events employs a "null point" which is the target for an "average but not exceptional" beer. As I have indicated in the past, we still use a 20 point scale with 1/2 points allowed (for a range of 40 points). Each area of evaluation has a point range indicated with the null point identified. We evaluate the following areas: Appearance 0-3 points (null = 2) Aroma 0-4 points (null = 2) Flavor/Body 0-10 points (null = 6) Memorability/Overall impression 0-3 points (null = 2) A "null point" beer (12 points) is one which has no significant flaws, generally fills the requirements of the style, but is unexceptional. It provides a standard similar to the 0 of the +/- scale in that you can determine how good or bad the judge thought your beer was relative to where the final score is to the null point. RW... -------------------------------------- From JudgeNet Tue Mar 17 17:14:48 1992 Subject: proposed scoresheet -- good, bad or ugly?? Here is a proposed scoring system based on suggestions and comments from this mailing list. Tell me what you like or dislike about it. Jay Hersh helped me because he likes the idea of +/- scores. happy St. Patrick's day but no green beer for me, Bob Devine ------------------------------------------------------ 1. Intro There has been a lot of discussion about changes needed in the current AHA score sheet. While there in not universal agreement on the problems, here are some complaints: - the bottom end of the range (0-19) is practically useless because all beers in that scoring range are rated as undrinkable so it leaves only 30 points out of the 50 as usable - the "good" and "drinkable but flawed" ranges are only 5 points even though lots of beers fall in that range - a range of 0-50 is not as "natural" as 0-100 2. Goals Here are some of the goals we used for this scoring system. - use the 100 point range - use +/- scores for each section - allow a built-in "supportive" score for bad beers - have wider point ranges for each certificate award range - don't fundamentally change the existing score sheet 3. Proposal After mulling over several different proposals, the one that incorporates the changes is the following: AWARD CATEGORIES EXEMPLIFIES STYLE PROBLEM LEVEL 90-100 Excellent well none or little problems 75-89 Very Good well little problems 60-74 Good okay medium problems 50-59 Drinkable poor medium problems < 50 Problem (doesn't matter) big problems SCORE SHEET Bouquet/aroma (-7..7) ___ Appearance (-4..4) ___ Flavor (-14..14) ___ Body (-2..2) ___ Drinkability / Overall (-7..7) ___ Add +66 TOTAL ___ This scoring proposal has a range of +/- 34 points from the mean of 66. A +/- 50 range seemed overly broad and one of +/- 25 too narrow. The maximum high score is 100. The maximum low of 32 should not be given because any scores below 50 can be equally expressed as a `49' or just have the total marked as "no score". (I think a "no score" marking is valid because at last year's 1st round Charlie Papazian had a still mead that gushed explosively from the bottle so that there was not much remaining in the bottle to judge.) The rules for scoring is to consider 0 points for a section to mean "beer has no significant flaws, generally fills the requirement of the style, but is unexceptional" (in the words of Russ Wiggleworth as he described the Maltose Falcon's concept of a `null point'). This is a useful concept! Points are awarded based on how the judge perceives the beer compares to the null point -- a 0 is in the middle of the range. Major flaws should result in very negative points; superior style representation should mean very positive points. And of course the hard part of judging is the middle area: How much diacetyl is too much? Is the beer too hoppy for style? Is the color too dark? The points given to these sort of "how good is good/how bad is bad" questions will be slightly positive or negative depending upon the style of the beer. 4. Discussion This scoring system tries to answer the concerns of people without radically changing the current tried-and-true system. As Steve Stroud pointed out, a 100 point system helps prevent one section from causing an out-of-proportion change to the final score. This proposed scoring system requires that high, positive points be awarded in more than one section for total score to be in the "very good" or "excellent" category. As an example, consider a pale ale that tastes great, has the appropriate hopping levels, carbonation, color, etc. but is way too cloudy. The only negative score should be in appearance with positives for everything else. In the negative direction, consider how an abused, old bottle of Pilsner Urquell would be scored. Negative points would likely be given in the aroma, flavor, and drinkability sections so that the total score would fall into the "problem" category even though it may have scored well in the appearance and body sections. One concern was the possible reluctance of some judges to award negative points. The inclination might be that "when in doubt, give a 0". Therefore there is a bias to median scores so that only very good or very bad beers produce a score towards the outside of the score range. -------------------------------------- From JudgeNet Tue Mar 17 23:14:05 1992 Subject: Re: proposed scoresheet -- g Reply to: RE>proposed scoresheet -- good I Like It! This covers the best of the scoring styles that have been discussed here. All are incorporated well. On first glance the only suggestion I have would be to limit the range for flavor to +/- 12 and expand the range for overall impression to +/- 9. I don't see the need for so much more range for flavor. The bigger "Drinkability" range gives more latitude for judges to slight scores for "out of style" entries and reward beers they really like. I have the same complaint with the current AHA sheet, it is just my particular bias. Oh yeah, I'm a San Andreas Malt, not a Falcon... (;-) RW... -------------------------------------- From JudgeNet Wed Mar 18 13:20:21 1992 Subject: Re: proposed scoresheet -- good, bad or ugly?? Sorry guys, but I am strongly opposed to the idea of negative scoring, even if the sheet is set up to still give a positive final score. I prefer the way it is done now, ie. give fewer points for a substandard flavor, aroma, etc. I can see the logic in expanding to a 100 point system, but I would not support it if negative scores are used. I would go so far as to refuse to use the style sheet in competition, or decline to judge in a competition that used them. On another note, how do I get my classification changed to 'Recognized' for this list? I passed my BJCP exam, and in fact I am nearly up to the number of points necessary for the next level (certified?). Mike Zulauf zulauf at orbit.colorado.edu -------------------------------------- From JudgeNet Wed Mar 18 13:29:58 1992 Subject: SCORE SHEETS, WHEAT BEER I LIKE THE IDEAS FOR A NEW 100 POINT SCORE SHEET. NOW, TO CHANGE THE SUBJECT. I OBJECT TO THE INCLUSION OF THE AMERICAN WHEAT STYLE IN THE AMERICAN LIGHT LAGER CLASS ACCORDING TO THE AHA DESIGNATIONS. I BELIEVE AMERICAN WHEAT SHOULD BE CLASSED WITH THE OTHER WHEAT BEERS IN CLASS 24. AMERICAN WHEAT IS TYPICALLY AN ALL-MALT BEER, AND THE STYLE HAS BEEN POPULARIZED BY HOMEBREWERS AND MIRCOS. YET ITS PLACE IN THE AM. LIGHT LAGER CLASS MEANS IT IS JUDGED AGAINST AMERICAN CORN BEER, AMERICAN RICE BEER, DIET, AND DRY, --- THE INDUSTRIAL BEER STYLES. THE BIG BREWERS DON'T EVEN MAKE WHEAT BEER. I WOULD RATHER JUDGE AM. WHEAT BEER AS A SUBCLASS WITH THE OTHER WHEAT BEERS. ANY COMMENTS FROM JUDGELAND? TOM ALTENBACH -------------------------------------- From JudgeNet Wed Mar 18 13:49:14 1992 Subject: who's who Due to the publicity generated by the Study Guide, we have several new subscribers on the list. I figured it was about time to post the membership list again. ### BEER JUDGE MAILING LIST ### # # Honorary Judges # uunet!att!drutx!homer Jim Homer - honorary # # Master Judges # # # National Judges # chuck Chuck Cox - national mikef Mike Fertsch - national uunet!hersh at expo.lcs.mit.edu Jay Hersh - national uunet!darrylri at microsoft.com Darryl Richman - national uunet!stroud%gaia at polaroid.com Steve Stroud - national # # Certified Judges # sheri Sheri Almeda - certified uunet!baughmankr at conrad.appstate.edu Kinney Baughman - certified uunet!clarke71 at snycorva.bitnet Dwight Beebe - certified uunet!oran at turnip.bally.com Oran Carmona - certified uunet!lcarter at claven.idbsu.edu Loren Carter - certified uunet!steved at longs.lance.colostate.edu Steve Dempsey - certified fitz Jim Fitzgerald - certified uunet!sequent!techbook!gummitch Jeff Frane - certified semantic!judge-in Bob Gorman - certified uunet!mlh at cygnus.ta52.lanl.gov Michael Hall - certified uunet!hopduvel!brewmaster at linac.fnal.gov John Isenhour - certified uunet!malodah at pbmoss.pacbell.com Martin Lodahl - certified uunet!srussell at msc.cornell.edu Stephen Russell - certified uunet!strasser at raj5.tn.cornell.edu Tom Strasser - certified uunet!rad_equipment at radmac1.ucsf.edu Russ Wigglesworth - certified # # Recognized Judges # uunet!altenbach at nsspb.llnl.gov Tom Altenbach - recognized uunet!tony at spss.com Tony Babinec - recognized uunet!gcb at fc.hp.com Glenn Colon-Bonet - recognized uunet!u52983 at uicvm.uic.edu Roger Deschner - recognized uunet!stevie at spss.com Steve Hamburg - recognized uunet!e3bsr at psuorvm.bitnet Doug Henderson - recognized uunet!att!ihlpl!korz Al Korzonas - recognized uunet!jelj at cornella.cit.cornell.edu John Lenz - recognized uunet!tedm at hp-pcd.cv.hp.com Ted Manahan - recognized uunet!resch at craycos.com Dave Resch - recognized uunet!ross at mscf.med.upenn.edu Andy Ross - recognized uunet!wrs at gozer.mv.com Bill Slack - recognized uunet!72240.2510 at compuserve.com Eric Webster - recognized uunet!zulauf at orbit.colorado.edu Mike Zulauf - recognized # # Apprentice Judges # uunet!polstra!larryba Larry Barello - apprentice uunet!peterb at cs.cmu.edu Pete Berger - apprentice uunet!michael at wupsych.wustl.edu Mike Biondo - apprentice uunet!blommel at admin1.usask.ca Laura Blomme - apprentice uunet!embreed at sfovmic1.vnet.ibm.com Emily Breed - apprentice uunet!bli at psuvm.psu.edu Jeff Brendle - apprentice uunet!dwbutler at mtus5.cts.mtu.edu Daniel Butler-Ehle - apprentice uunet!cary at nas.nasa.gov Matt Cary - apprentice uunet!condof at cgsvax.claremont.edu Fred Condo - apprentice uunet!bwc at icd.ab.com Barry Cunningham - apprentice uunet!jdecarlo at mitre.org John DeCarlo - apprentice uunet!tdenny at rigel.cs.pdx.edu Tom Denny - apprentice uunet!devine at cookie.enet.dec.com Bob Devine - apprentice uunet!donham at super.enet.dec.com Perry Donham - apprentice uunet!drumm at cnsvax.uwec.edu Dan Drumm - apprentice uunet!easterm at ccmail.orst.edu Mark Easter - apprentice uunet!tee at cray.com Tony Ernst - apprentice uunet!gal2 at midway.uchicago.edu Jacob Galley - apprentice uunet!76702.764 at compuserve.com Robin Garr - apprentice uunet!jason at gibson.sde.hp.com Jason Goldman - apprentice uunet!green at hpmtaa.lvld.hp.com Bob Green - apprentice uunet!brew at devine.columbiasc.ncr.com Jim Griggers - apprentice uunet!joshua.grosse at amail.amdahl.com Joshua Grosse - apprentice uunet!hall at buffa.enet.dec.com Dan Hall - apprentice uunet!henchal at detrick-emh1.army.mil Erik Henchal - apprentice uunet!jeorg at chs.com Jeorg Houck - apprentice uunet!kermit at horus.cem.msu.edu Kermit Johnson - apprentice uunet!jones at decux.cmp.rpi.edu Ron Jones - apprentice uunet!rich at bedford.progress.com Rich Lenihan - apprentice uunet!loc at bostech.com Roger Locniskar - apprentice uunet!bmartin at iit.com Bryan Martin - apprentice uunet!mason at habs11.enet.dec.com Gary Mason - apprentice uunet!klm at gozer.mv.com Kevin McBride - apprentice uunet!rob at maccs.dcss.mcmaster.ca Robert McDonald - apprentice uunet!tmitchel at bbn.com Tom Mitchell - apprentice uunet!mece1xg at jetson.uh.edu Purna Murthy - apprentice uunet!bmyers at garnet.berkeley.edu Brian Myers - apprentice uunet!rcm at col.hp.com Rick Myers - apprentice uunet!mnolan at lpl.arizona.edu Michael Nolan - apprentice uunet!reeves at essdp1.lanl.gov Geoff Reeves - apprentice uunet!roody at whzguy.shr3.dec.com Greg Roody - apprentice uunet!arf at ddsw1.mcs.com Jack Schmidling - apprentice uunet!schultz at admin1.usask.ca Robert Schultz - apprentice uunet!chris at asylum.gsfc.nasa.gov Chris Shenton - apprentice uunet!70740.1107 at compuserve.com James Spence - apprentice uunet!gak at corp.sun.com Richard Stueven - apprentice uunet!kswanson at nwu.edu Kurt Swanson - apprentice uunet!tinsethg at ucs.orst.edu Glenn Tinseth - apprentice uunet!gdtms at garlic.lcs.mit.edu Greg Troxel - apprentice uunet!homebrew at tso.uc.edu Ed Westemeier - apprentice uunet!swh at ll.mit.edu Sarah White - apprentice ## ===== Chuck Cox chuck at synchro.com -------------------------------------- From JudgeNet Wed Mar 18 17:30:54 1992 Subject: Re: proposed scoresheet -- good, bad or ugly?? I confess to not having followed this discussion on judging forms as closely as I should, so apologies if I sound horribly uninformed. This most recent suggestion, however, with its +/- and 0=normal may be statistically interesting. From the perspective of this judge, on the other hand, and as a homebrewer--if it takes all those paragraphs to explain to _us_, who's going to explain it all to the entrants? The virtue of the current system--or a simple 100 pt. system--is that it's pretty straightforward, and it's simple to explain. I realize a few people are putting a lot of effort into this, but it seems to me an attempt to apply hard objective structure to an inescapably subjective analysis. Which is to say, I don't believe _any_ point system is going to be free of defects and we ought to apply Occam's Razor and settle for simplicity. --Jeff Frane -------------------------------------- From JudgeNet Wed Mar 18 17:52:28 1992 Subject: Re: SCORE SHEETS, WHEAT BEER hmm. IMHO the "American" wheat beers dont exhibit any of the characteristics of "classic" wheat beers. With them wheat is simply used as an adjunct to lighten the flavor. This and their typically lower alcohol content are what make them more comparable to the american light lager, and is probably the basis for the AHA's grouping (god do I hear myself correctly?? I'm supproting the AHA on something :-)... Myself and many other judges do consider many American Wheat beers to be bastardizations, and not true wheat beers. Rather than pollute what is a relatively well defined style solely because wheat is present as an adjunct would be IMHO a mistake.. -JaH -------------------------------------- From JudgeNet Wed Mar 18 18:01:09 1992 Subject: Competition Report Well, I haven't heard any traffic here from the other participating judges, but I thought I give my 2 cents on how the judging this weekend went with the new 100 point system. To refresh memories, the contest was the IBU St. Patrick's Stout and Porter Contest last Saturday run by Dwight Beebe. As per Steve Stroud's suggestion, the points on the sheet were set as: aroma 20 appearance 15 flavor 35 body 10 overall 20 total 100 And the scoring guide was defined as: Excellent 90-100 An Excellent representation of style Very Good 80-89 A very good representation of style, has minor flaw(s) Good 70-79 A good representation of style, has significant flaw(s) Drinkable 60-69 A drinkable beer, but serious flaw(s) distinguish it from the style Problem 50-59 A beer with serious flaw(s) which prevent it from being drinkable, regardless of style. >From a personal standpoint, I have to say that I found the system easy to adapt to, in that you didn't have to worry about giving a beer too many points in a given category, such that it's total score was not in line with the scoring guide. I perused the overall scores from all the judges, and think the scores as a whole were lower than I would have thought. I think this is probably the result of working with judges who did not realize that due to the scoring guide, half the points in a given category were virtually freebies, so an completely inappropriate aroma would still wind up with 10/20 points. This should just take some getting used to though. I didn't get to talk to many judges but the thing was kind of sprung on them, and they didn't grumble much...So I'll take it a a positive feedback of sorts. I did speak with Pat Baker and he seemed semi-positive about the system, although he was more of the mold that most any system could work. He like the idea of giving a wider range to prevent beers from recieving very similar scores on the low end of the scale. He also agreed the 100 point system would be more comprehensible to the average person entering a contest. He did express concerns over the difficulties of correctly adding larger number than the judges are used to. Actually I found this probably the hardest part about the scoresheet, and at least at first, a little distracting from my judging. My way around it was to keep 2 columns on the sheet, the points awarded in each section, as normally done, and in the margin next to this I kept the points deducted for each section. Thus for most beers, the deducted points were smaller and could be counted easier, and subtracted from 100 to get the score. The system was effective in my flight in preventing beers from all being "clumped" together. There was a small clumping of beers in the 78-82 range, but this should be expected for good beers with some stylistic flaws. Other than that there wasn't much grouping at all (lucky for me, not even in the problme range!). Finally I thought I would congratulate Dwight Beebe on a job well done. The competition had 76 entries in 5 categories, and went very smoothly, IMHO, even with the introduction of the scoring sheet (although the stewards probably had a tough time sharing 1 calculator in verifying scoring sheets). There were entries from New York, Connecticut, New Jersey, Wisconsin, and Georgia...and surprisingly enough, beers from all of those states placed. Results: BOS - Sweet Stout, Chris and Liz Stamp, (IBU), Rock Stream, NY Runner-up - Imperial Stout, Jim Link, (UB), Shelton, CT firsts: Robust Porter - Harry Clayton (UB/IBU), Seymour, CT Brown Porter - Steve McLaughlin (IBU), Orwell, NY Dry Stout - Gary Walther (Bellhops), Somerset, NJ Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Wed Mar 18 18:18:47 1992 Subject: Reply to "Re: proposed scoresheet -- good, bad or ugly??" (03/18/92 1 Mike Zulauf says: > Sorry guys, but I am strongly opposed to the idea of negative >scoring, evein if the sheet is set up to still give a positive final >score. I would say that it depends on how you look at the scores. For reporting scores to the person submitting the beer, an 8 out of 10 is much to be preferred to a 3 out of (-7 to +7). However, when addressing how to *judge*, the idea of a null point and adding or subtracting is useful in some sort of judges guide. It may sound primitive, but I think I find it easiest to give each area a letter grade (A,B,C) and translate that to an equivalent X out of Y. Internet: jdecarlo at mitre.org (or John.DeCarlo at f131.n109.z1.fidonet.org) Fidonet: 1:109/131 -------------------------------------- From JudgeNet Wed Mar 18 18:33:46 1992 Subject: Re: proposed scoresheet -- good, bad or ugly?? Mike Z. says > Sorry guys, but I am strongly opposed to the idea of negative scoring, even if the sheet is set up to still give a positive final score. and > I would go so far as to refuse to use the style sheet in competition, or decline to judge in a competition that used them. guess we'll miss ya Mike... :-) Care to provide any constructive comments as to why this merger of what Bob and I feel are the strong points of the +/- and 100 pt systems, rather than just stating your preference. Thanks, Jay -------------------------------------- From JudgeNet Wed Mar 18 20:12:58 1992 Subject: Re: proposed scoresheet -- good, bad or ugly?? Just a thought of the way to 'appease' both the +/- and all positive numbers camps from a poor college student who can't afford to go to where they give the beer judge exams. Why don't we make the form with a "number line" sort of way to register the 'score' so that if we say have 'body' which is +/-2 in the system now, you could do something like a 0-5 range, where '3' represents the "normal" score and then *we* could think of the other scores and representing either faults to subtract from the score on the item or merits to add to it, but the homebrewer would still see that familiar "100% scale". What do you think? Make it easier on all concerned to be able to say that, "hmm, this is a touch watery, maybe I will dock it one point, so that's a 2" or "my, but am I impressed with the full mouthfeel of this one, I'll give 'em two, that makes a 5"?? Just a suggestion... -Jeff B. PSU Homebrewer -------------------------------------- From JudgeNet Wed Mar 18 22:08:46 1992 Subject: Re: proposed scoresheet +/- (100) } Jay says ... } Care to provide any constructive comments as to why this merger } of what Bob and I feel are the strong points of the +/- and 100 } pt systems, rather than just stating your preference. I think that the +/- system you proposed based on a 100 point system is fine, but isn't it really just a method of judging? The way I see it, you can have a null point (like 12 is for the 20 point system) and add or subtract from that, or you could just determine a way to come up with the appropriate score. It's probably up to the individual judge how he/she prefers to do it. At least that's the way things are run now. Actually, I would say I've already used a variation of this: when I scored I started with a mindset of giving half the point total in each section for free, and added similar to the 50 point system from there. It's all in how you want to judge, IMHO. Just had to add my $0.02. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Thu Mar 19 12:39:28 1992 Subject: Re: proposed scoresheet & New thread Perhaps a couple more "real world" judgings using the 100pt system with & without the +/- twist would provide some user feedback as to what really works. New topic: How about some pointers (FAQ) to representative beers for particular styles? There are the (commercial) beers that are considered the basis for a style e.g. Pilsner Urquell - but there must be a reasonable set of beers that express the range of a style as well. I suppose it might be hard to locate good examples in the US (how does one find a reasonably fresh bottle of PU?). Is there already a list somewhere with this information? - Larry Barello uunet!polstra!larryba -------------------------------------- From JudgeNet Thu Mar 19 13:09:55 1992 Subject: Re: proposed scoresheet +/- (100) Tom Strasser says: > I think that the +/- system you proposed based on a 100 point > system is fine, but isn't it really just a method of judging? Yes it does do that but it does more. It makes explicit the rating of a beer to the mid-point for the style (ie, the null point). Many judges seem to already do the method of _mentally_ adjusting points by adding or subtracting from the middle of the point range. However that kind of information - how a beer relates to the null point - is never directly given back to the brewer. Bob Devine -------------------------------------- From JudgeNet Thu Mar 19 14:08:10 1992 Subject: Using the 100 point system Judges, I used the 100 point scoresheets this past weekend as a judge. In previous competitions, I had used both the AHA- and HWBTA- 50 point sheets and was somewhat accustomed to them, but not to the point I was seeing them in my sleep or anything like that. My observations: -- as Tom Strasser mentioned, scores seemed to be somewhat deflated. My theory is that this is psychological: it is easier to want to boost points when most beers are scoring 25-35 and easier to deduct when points are in the 65-75 range. (and we were all-but instructed to score bad beers in the 50s by the competition orga- nizer...Hi Dwight!) -- this effect might temper as judges get used to the system, but I think to a certain degree it won't. Why? Judges that were previously reluctant to assign low scores will to a certain extent be calmed by the fact that the overall score is now 50 instead of 10, say, and will tend to lower their scores for non-drinkable and out-of- style beers. I know that I put some beers in the 55-58 range whereas before I had never scored a beer below 19 -- and it didn't bother me. (I did it and I'm glad :) -- the better beers should distinguish themselves more with this system. With scores tending to deflate overall, the better beers will fall in the 80-100 range (I figure that in the old system, better beers fell into the 35-50 range). Gives us more range to work with. I don't envision that this change will be "revenue-neutral" even though we would like it to be. Unexpected effects will occur; we will have to revise our expectations to some degree. And I don't pretend to be able to foresee what some of these changes will be. Just that I don't believe that you could take scores under the old system, add 50 and get the same score (roughly) from a different judging panel using the new system. Even after instructing judges that this is how it's supposed to be! Overall, I think that the system is a tad more difficult to work with, no surprise given the doubling of points available to allot. But the apprentice judges and stewards I polled (a statistically insignificant 8 people) agreed with the sentiment that a 100 point scale is more intuitive. Brews to ya, STEVE -------------------------------------- From JudgeNet Thu Mar 19 15:34:58 1992 Subject: Re: Competition Report Jeff Frane says: >From the perspective of this judge, on the >other hand, and as a homebrewer--if it takes all those paragraphs to >explain to _us_, who's going to explain it all to the entrants? The explanation was for the benefit of this group, to adress issues that had been brought up during the discussion you haven't been following. It would be a trivial matter to print up the score sheets to indicate the base score is 66, and show the +/- values for each section. I don't think it woyld take a rocket scientist to catchon. Tom S. indicates that his perception of a trial with the 100 pt score sheet is tat it will take some getting used to. This is natural, and the same thing will hold for any score sheet. It took getting used to judgnig in the first place right?? Why do people apprentice now?? Because working with the sheets to assign an apprpriate score takes a little getting used to. Tom then says >My way around it was to keep 2 columns on the sheet, the points awarded >in each section, as normally done, and in the margin next to this I kept >the points deducted for each section. Thus for most beers, the deducted >points were smaller and could be counted easier, and subtracted from >100 to get the score. Hmmm, you used a system where you deducted points. Radical idea Tom, I think maybe you're on to something :-) :-).... Jeff B.'s number line idea is esentially what I try to apply now. what drove the idea of +/- is that some judges don't like to award points for a beer with flaws. If a beer has an onvious off aroma why should I credit it points?? Rather I should detract them IMHO. If a beer has a bland but unflawed aroma then I owuld want to give it few or no points, if a beer has a very appropriate aroma, it should be credited. This is a simple concept, and could be easily communicated on the score sheet in the bottom area where the ranges are defined with a simple statement such as "Within each section points are awarded for characteristics appropriate to style, and deducted for off-aromas or other flaws" Tom also points out that using a "null point" is and adding subtracting from that is essentially the same. I don't think this is quite so, since what Bob had proposed was using a starting score of 66, and working +/- 34 around that. If I try to do this in a 100 point scale my scores will be skewed from someone using the 100 pt without this. If I am self-consistent , that is OK I guess but presently there is a lot of pressure for all judges to be within some minimal range of each other. If 2 judges are using 2 different approaches on the same score sheet (say I just silently apply mine & Bob's approach) it is likely my scores while self-consistent will not be in this range (also because the good, very good, etc. ranges are defined differently). Gee this is such a lively topic, my tendonitis is killing me... -JaH -------------------------------------- From JudgeNet Thu Mar 19 18:15:01 1992 Subject: Re: Using the 100 point system } Steve Russell says... } -- this effect might temper as judges get used to the system, but I think } to a certain degree it won't. Why? Judges that were previously } reluctant to assign low scores will to a certain extent be calmed } by the fact that the overall score is now 50 instead of 10, say, } and will tend to lower their scores for non-drinkable and out-of- } style beers. I know that I put some beers in the 55-58 range } whereas before I had never scored a beer below 19 -- and it didn't } bother me. (I did it and I'm glad :) For the same reasons I thought a beer should not be scored below a 19, I don't see the rationale for giving a beer less than a 59. At least in the 50 point system it could be said that you couldn't find the place to *give* 19 points, but if you're already giving 50 points, I see no reason to quible over a a couple more...be encouraging! It won't effect the competition results any. } -- the better beers should distinguish themselves more with this system. } With scores tending to deflate overall, the better beers will fall } in the 80-100 range (I figure that in the old system, better beers } fell into the 35-50 range). Gives us more range to work with. While I reported this as a general result of an initial test, I think the judges should learn to using thee scoring guide as it's defined. I'm sure it will take time for a judge to adjust, but when experienced I see no inherent reason the scoring should not conform to the guide. An example would be giving a beer a 10/20 for aroma. This seems middle of the road to an old 50 point judge, but would effectively correspond to giving 0/10 in the old system, so the judges really will need to adjust some, but it won't happen overnight. I would also want to discourage the expansion of the higher end of the scale as suggested because at least at the contests I've been at, there is no need. In a single flight you'll be lucky to get 3 beers in the 40 point range, and with ten points to play with there, why would it be necessary to expand the range? Thus I think the judges should be encouraged to adjust to the scoring guide. } I don't envision that this change will be "revenue-neutral" even though } we would like it to be. Unexpected effects will occur; we will have to } revise our expectations to some degree. I agree unexpected effects will occur, but I don't think these *have* to be representative of what experienced judges would do with a system. When each of us first judged with any system, there were inevitably "unexpected effects", but good judges should evaluate their scoring in a system, and adjust to fit the scoring guide. Thus I would urge judges to be careful when initially using this sheet, because there will be initial tendencies to stray from the scoring guide. } Just that I don't believe } that you could take scores under the old system, add 50 and get the same } score (roughly) from a different judging panel using the new system. } Even after instructing judges that this is how it's supposed to be! Not on the first trial, and not ever for the good and drinkable ranges where this method would not apply, but I think you should expect the judges to come to follow the guide with experience. } Overall, I think that the system is a tad more difficult to work with, } no surprise given the doubling of points available to allot. But the } apprentice judges and stewards I polled (a statistically insignificant } 8 people) agreed with the sentiment that a 100 point scale is more intuitive. I disagree. I find the addition more difficult, but I think that is more than balanced by the increased flexibility you get from awarding a larger number of points. For this reason I find the necessity for "bottom up" scoring decreased, and thus you can fill in point totals as you go rather than waiting to taste the beer in fear you may award to many points early, requiring a major scoring revision. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc2.bitnet -------------------------------------- From JudgeNet Thu Mar 19 18:57:40 1992 Subject: some comments on the +/- scoring system After having read Bob Devines posting on the good/bad/ugly +/- scoring system and mulling it over a bit, I have a few comments. There are two things that bother me *a lot* about it: 1) I think that the proponents of the +/- system are *still* missing the psychological impact of zero or negative scores for each attribute. Judges won't like to give, and brewers won't like to receive, a "-" score for an attribute that is in the good range (by Bob's own definition, a good beer is 60-74, which means that a beer could receive a total of -6 [lots of zeros and a few minus numbers] and still be called good). It just isn't intuitive and would, in my opinion, take a MAJOR re-education of the judges and brewers to make work. Additionally, the +/- proponents have made a big deal out of that system having a "null" point , and have suggested that a brewer can learn a lot about their beer based on its ranking relative to the null. I admit that I don't understand how this really helps the brewer. If you tell me that the +/- system has a null point at 66 (middle of the good range), and I tell you that the 100 point scale has a null point at 75 (middle of the good range), where is the advantage? It's the comments that really give the brewer feedback, not the score. 2) This may sound silly, but I think that it is a MAJOR stumbling block for the +/- system: The judges need to add + and - scores from the attributes to come up with the final score. Sounds easy, doesn't it? Yet I predict that there will be a lot more mistakes when the judges have to total up the score sheets. Tom Strasser pointed out that he had trouble just adjusting to the addition with the larger point range in the 100 point system. I think that negative numbers mean math errors. An example of what I'm talking about: Here's a beer. I give it aroma: -1 appearance 3 flavor -8 body 2 overall -5 then, +66 to total Quickly, what's the final score? It's not hard, but takes a bit longer, doesn't it? And you're sober, I hope :-). Imagine if this were the fourteenth entry in the class of strong ales that you're judging. I really believe that the totalling up of scores would be a big pain in the butt with the +/- system. Of course, the proof is an actual demonstration. Steve Russell has told us that the unexpected may occur and I could be way off base. The 100 point scale seems to have a relatively successful maiden voyage. Can Bob and Jay convince a competition to do a trial run on the +/- system and report back on how well it worked? Steve Stroud -------------------------------------- From JudgeNet Fri Mar 20 14:16:10 1992 Subject: Reply to "Re: Competition Report" (03/19/92 17:33:18) Jay Hersh says: >what drove the idea of +/- is that some judges don't like to award points >for a beer with flaws. Yes, but how this is implemented relates to your later statement. >but presently there is a lot of pressure for all judges to be within some >minimal range of each other. If 2 judges are using 2 different approaches As a lowly apprentice, the need for judges to be within some minimal range of each other seems obvious, yet difficult to make happen. As I tried to point out earlier, but must have done so badly, the approach a judge uses makes a big difference. Using the +/- scoring method more-or- less forces the judges to use that method as well. Using the X out of 10 approach currently allows judges to start at a null point and add/subtract, or to only subtract points from the 10 possible, or to give it a letter grade and translate that (oooh, a D-, that means a 6 out of 10), or some other method. Then any pressure seems to come when the total scores differ wildly. In other words, it seems to me that you *have* to standardize on a method for giving the points in each section as well as a method of scoring to get any consistency. An advantage of the +/- is that it does both in one. Internet: jdecarlo at mitre.org (or John.DeCarlo at f131.n109.z1.fidonet.org) Fidonet: 1:109/131 -------------------------------------- From JudgeNet Fri Mar 20 17:25:34 1992 Subject: Re: some comments on the +/- scoring system >Quickly, what's the final score? gee, add all the negatives and all the positives separately -14 + 5 = -9 then add the total to the base. -9 + 66 = 55 Duh!!! Have any of you actually graduated High School??? I can't believe the fuss you make over adding and subtracting. Do you balance your checkbooks, or are you all members of Congress??? I've never seen so much bandwidth wasted over a non-argument (well almost never... :-) Lots of folks have talked about what the message you send to a brewer is. Well what kind of message is sent when you write that a beer has a phenolic off aroma/flavor, and then go ahead and award it points anyway. What could be more clear than deducting points for a flaw, and awarding for a positive attribute. That sends a direct message to the brewer that yes, there IS a problem here. The judge understands, and the brewer understands that this component of the beer has a problem! Awarding points, even though it may be a smaller number, where a problem exists sends an ambiguous message. Was this a problem, or not. Comments say so, yet I still got points, so maybe it isn't so bad??? There is also a major fallacy in comparing the 100 point system to grade school scoring. In grade school when you got a 55 you failed the exam, BUT you earned the 55 points you got, you DID know something, and every point you were awarded was an earned point. This is not the case with the 100 pt system, nor is it the case with modified 100 +/-, though it was the case with the original +/-, where every point was earned. A few observations 1) Any new system is going to take getting used to. IMHO the difference in accomodating to one vs. the other is trivial. 2) Adding and subtracting obviously needs to be aded to the certification test since so many of you have a problem with it. If you can't do this perhaps you SHOULDN'T be recognized. 3) If you want to provide unambiguous feedback on a beer, then award points for positive charcteristics, and penalize for undesirable characteristics. >The 100 point scale seems to have a relatively successful maiden voyage. Can >Bob and Jay convince a competition to do a trial run on the +/- system and >report back on how well it worked? What's the upcoming schedule look like. Anyone willing to bite the bullet and give it a try?? The only local competition I know around here is the 1st round national, and I don't think they want us experimenting with the scoresheets:-) -jay -------------------------------------- From JudgeNet Fri Mar 20 19:23:45 1992 Subject: Son of "100 point system" Jay sez: "it is a sad commentary on our society when people find it more difficult to add points in a 0-100 range vs. adding in a 0-50 range. folks addition is addition, either you can do it or you can't. personally I think you've all gone a little mush in the head... too many homebrews :-)... Wrong, Jay. Math is not a "yes I can" or "no I can't" process. It is a very brain-hemisphere concentrated event and what seems trivial to you perhaps may be much harder for someone else. yes, even addition. The point is that there are all kinds of abilities when it comes to math and when someone tells you that it is easier to use one system than another (for them), you should accept it at face value. Personally, I think that it probably IS easier to do the math in a 50 point system (most of the time it involves adding 5 single digit numbers) than in a 100 point system (adding 5 2-digit numbers). Of course, that person that you're smurking at for taking a bit longer to do their addition may have a much better degree of sensory perception than you or they may have a much easier time of expressing themselves verbally on the scoresheets or whatever. The point is we're all different and should be aware of it, but should also accept it. A hand calculator, a flashlight, and a mechanical pencil may become the utensils de rigueur of the BJCP in the '90's. :-) Steve Stroud -------------------------------------- From JudgeNet Sat Mar 21 07:13:17 1992 Subject: Re: Son of "100 point system" Stiv Stroud comments: > A hand calculator, a flashlight, and a mechanical pencil may become the > utensils de rigueur of the BJCP in the '90's. :-) That was the eighties, Steve. In the nineties, we need laptop computers, portable spectrophotometers, and pocket Postscript printers. Already we have style descriptions that state acceptable alcohol ranges; without sophisticated analytical equipment, how is a judge to decide if the beer is appropriate for style? "Hmmm, that tastes like 13.1% alcohol, the style description says 13.0 is the maximum. Using the +/- system, I'll deduct 0.4 points." Personally, I prefer a 1000 point system, with the restriction that all scores be integers. Integer math is much easier! -- Mike Fertsch Internet: mikef at synchro.com Internet: synchro!hopfen!mikef at uunet.uu.net Wortnet: synchro!mikef -------------------------------------- From JudgeNet Sat Mar 21 11:13:56 1992 Subject: The new math? Greetings rocket scientists, Jay says: >Date: Fri, 20 Mar 92 16:33:41 EST >Message-Id: <920320.255.judge at synchro.com> >From: Jay Hersh >Subject: Re: some comments on the +/- scoring system > > >>Quickly, what's the final score? > >gee, add all the negatives and all the positives separately > -14 + 5 = -9 >then add the total to the base. > -9 + 66 = 55 > >Duh!!! Have any of you actually graduated High School??? >I can't believe the fuss you make over adding and subtracting. >Do you balance your checkbooks, or are you all members of Congress??? >I've never seen so much bandwidth wasted over a non-argument (well >almost never... :-) Where is Rob Bradley when we need him. So Jay, what's the scoop, is this the new math, or what? :-) I think that you may have unwittingly made the strongest case so far for not adopting the +/- system. Assuming that you had time to carefully craft this post, I wonder what sort of scores you'd arrive at in an actual judging situation ;-), perhaps some of the math fuss is actually warranted. Ooogy wawa, Dr. John -------------------------------------- From JudgeNet Sun Mar 22 14:23:15 1992 Subject: Re: Son of "100 point system" Guess we're pretty tired of this subject. I propse the following. Let's find someone to do a test with a 100 pt +/- system like Bob has proposed. I personally will use whatever system is adopted, but I oppose the vanilla 100 pt. system. I think it adds very little over the current 50 pt system and is not worth the change. Whichever system is adapted I will continue to be displeased if it is not one that addresses what I consider to be the primary issue, the ability to easily diferentiate positive from negative traits by a system of awards and penalties. Let's table the discussion until a trial can be done. -Jay -------------------------------------- From JudgeNet Mon Mar 23 08:13:55 1992 Subject: Reply to "Re: some comments on the +/- scoring system" (03/20/92 19:30 Jay Hersh said: >In grade school when you got a 55 you failed the exam, BUT you earned the >55 points you got, you DID know something, and every point you were >awarded was an earned point. I'm not sure what your point (no pun intended) is here, Jay. Sure, you got a 55 on a history test. You got your name right (15 points), you understood that asking when Columbus landed in America called for a date (OK, so you put 1911, but you knew it was a date), etc. Teachers don't go giving grades from 0-55 for the Fs, any more than beer judges do. OK, how many points should the brewer get (in the same analogy) for actually making *beer* (instead of cider, say)? 15 points? What about for correctly understanding that yeast is needed? 10 points? Before long, you get some low end below you might as well not give any score because the entry is so whacko. >3) If you want to provide unambiguous feedback on a beer, then award points >for positive charcteristics, and penalize for undesirable characteristics. This is the concept of a null point, and has nothing to do with requiring that null point to be zero. It's like telling people: "If you get a C, it has no serious flaws, but nothing particularly great, if you get an A, it is a very good beer, if you get an F, it has serious flaws". What is easier to understand than that? Just like in a test in school, identifying the specific errors is the only truly *helpful* feedback. Internet: jdecarlo at mitre.org (or John.DeCarlo at f131.n109.z1.fidonet.org) Fidonet: 1:109/131 -------------------------------------- From JudgeNet Mon Mar 23 15:14:50 1992 Subject: Son of son of 100 point system >In the nineties, we need laptop computers, portable spectrophotometers, and >pocket Postscript printers. Actually, we could get rid of the BJCP judges entirely by requiring that each competition organizer have a GC/Mass Spec available on site. The AHA could then submit an ideal gc "fingerprint" for each style that would be used as the standard that each entry would be judged against. An onboard computer would analyze a sample of each entry and determine how close it was to the "null" point for the style, assigning + or - points according to the %area of each peak in the trace. An added benefit of such a procedure, of course, would be that 12 oz bottles of your finest would no longer be needed. 100 microliters would probably be all that was necessary. In addition, accuracy of the analyses to 6 decimal places would virtually ensure that ties wouldn't happen. We could use the 0 - 1.000000 scale of judging (something I've championed all along), stop the ongoing debate about various scoring systems, get back to brewing and drinking homebrew. Sounds good to me :-) Stiv The danger to such a system, of course, is exemplfied by Kevin's accurate observation of the *undetected* single bit error that occurred in the integer add unit of Jay's computer. Two onboard, double sampling Fourier Transform computers should minimize the chance of such an occurrence. Having a service engineer present at all times would be required for competition certification. -------------------------------------- From JudgeNet Wed Mar 25 18:17:09 1992 Subject: forwarded from C. Papazian 3/25/92 From: SDI::IN%"72210.2754%CompuServe.COM at prdnet.polaroid.com" "Charlie Papazian/Boulder" 25-MAR-1992 16:42:00.21 To: IN%"STROUD%GAIA at leia.polaroid.com" CC: Subj: SCORING SYSTEMS Steve please forward this message into the judge forum: Charlie I finally had a chance to review the large file you sent me with regard to scoring. Here are a few comments I'd like to add. They are meant to be offered as part of discussion, not as any official policy statements from the AHA. There are several areas that you all have discussed. I'm only addressing a few that seem to have evoked some kind of response from me. RE: No beer deserves below a 19 As a judge myself, I feel that if a beer is that much out of style and that badly contaminated, then the brewer deserves an honest appraisal, even if in the evaluation of the judge it means a score of below 19. BUT, whenever I encounter a beer that has a great deal of problems I simply do not assign it a numerical score. AND I explain to the brewer why, in that the beer such and such problems and rather than depressing the brewer with a very low score, I will go into even greater detail describing what I have perceived AND suggestions as to how things can be remedied. I'm with many of you, in that giving a brewer a very, very low score is depressing and defeats the purpose. Giving a score of zero is never justified, I think. A zero score is an impassioned score given by a judge, not a constructive statement. I advise in the case of gushers, and really wretched (pardon the expression) brews, that again no score be given and simply explain to the brewer what you perceive and what the remedies may be. Don't get pissed off or impassioned just because you feel, "How dare this brewer have me judge a gusher, or have me judge such a wretched beer" Keep your passion out of it as much as possible. RE: + - scoring system As I mentioned earlier, the AHA used to have a positive and minus points scoring system. We used it for about 2 years or so, but it was criticized enough and unpopular enough that we responded by reevaluating the system and discontinuing it in favor of our present 50 point system, Which too, has gone through a few transitions, due to the constructive criticism and suggestions we have received. RE: 100 point system. I personally feel that a 100 point system MAY be justified in light of how the AHA scoring system is used. It is used in some rather large competitions, such as the national. And, for starters, simply doubling the present scale and evaluation criteria may better serve judges attempting to dilineate between several beers in a competition as large as the AHA nationals. Please keep in mind that my feelings are not what dictate how the AHA scoring system evolves. I would (or even someone else could, as you have through forums such as this) suggest ideas for consideration. I will run the 100 point idea through our competition people here in Boulder and Denver area, and as well request comments from our Board of Advisors and through various Computer forums such as this. Thanks for all the great conversation. Sorry I can't drop in on this forum every day, but perhaps one of you could summarize some of the thoughts and conversation, say on a monthly basis and feed it to me via E mail. Charlie Papazian, Boulder, Colorado -------------------------------------- From JudgeNet Fri Mar 27 00:14:13 1992 Subject: Re: forwarded from C. Papazi Reply to: RE>forwarded from C. Papazian Some clarification: A while back I made the comment that I had no problem awarding a "0" (zero) score to a beer which, for one reason or another (gusher or whatever), I could not sample and judge. I equate awarding that "0" with "no score". To me it is the same thing. It, along with as detailed an explanation as I can muster, is what goes back to the brewer. Do most of you suggest "no score" at the bottom of the sheet in stead of "0"? I agree with Charlie (as I hope we all do) that a low score or even a "0" should not be the result of some emotional reaction the judge might have to an entry's messy behavior. I think the judge should offer as much commentary as possible. The anecdotal part of the sheet is more important than the scores when it comes to getting information to the entrant. RW... -------------------------------------- From JudgeNet Fri Mar 27 17:14:39 1992 Subject: Re: forwarded from C. Papazi my standing preference has always been to do what Charlie says he likes to do in cases of massive problem beers, i.e. make lots of usefule comments but award no score. Unfortunately too many computerized competition organizers insist a score must be awarded to all entrants. I have more than once essentially been forced into giving a score for a beer that I thought was in need of diagnosis and not judging.. -JaH --------------------------------------