From JudgeNet Wed Jan 8 10:37:08 1992 Subject: Generic Category Subject: Generic Category Time:7:47 AM Date:1/8/92 Happy New Year! The San Andreas Malts are reviewing the California State Competition (which we run each year) and the subject of adding a "generic homebrew" or "no commercial equivalent" category has come up. I'd like to get some feedback from the rest of you as to experience with such categories and/or opinions on them. The concerns expressed so far have been: 1) A flat open category would be judged unevenly since the "bigger" beers, which judges tend to award with higher scores, would be mixed with the lower gravity ones. 2) When including such a category in a competition which also uses the more standard commercial categories and sub-categories how do you maintain the entries as non-commercial styles? Example: the other categories penalize entries which are "not in style" even though the beer may be well brewed and without significant flaws. Would you do the same in an open category if the beer was obviously identifiable as a Dry Stout or an IPA by the judges? 3) Judging such a category would require a different score sheet since the beer would be evaluated by different criteria than the standard categories are now. These are some of the basic comments which came out of our first discussion of the topic. We intend to go into greater depth and that's why I am looking for input from this forum. Thanks in advance! RW... Russ Wigglesworth CI$: 72300,61 |~~| UCSF Medical Center Internet: Rad Equipment at RadMac1.ucsf.edu |HB|\ Dept. of Radiology, Rm. C-324 Voice: 415-476-3668 / 474-8126 (H) |__|/ San Francisco, CA 94143-0628 -------------------------------------- From JudgeNet Wed Jan 8 11:52:59 1992 Subject: re: Generic Category This is one of the favorite topics amongst judges when discussing new categories for competitions. There will probably be quite a variety of opinions on the matter, but I get to state mine first cuz synchro is my machine! There really seem to be two different motivations for creating an open class; to provide a category for 'known' styles that don't have their own category, and/or to provide a category for beers that don't fit any known style. I tend to refer to these as 'other styles' and 'odd beers' respectively. It is very important that you print a clear description of what is expected in the category and what the judging criteria will be. For the 'other styles' category, it would be ideal if there was a list of what styles are acceptable. This list could be as simple as the AHA national categories. Require competitors to indicate the specific style, and inform the judges of this. Judging would then be done as usual, with each entry being judged as appropriate for the indicated style. Beers brewed to a style that already has a specific competition category, and beers that are not brewed to a 'known' style would be considered inappropriate. Obviously this category would benefit from judges with experience in a wide range of styles. An example: "OTHER STYLES: This category is limited to beers brewed to specific styles not already covered in this competition. Acceptable styles are limited to styles X through Y as described in the 1992 AHA National Competition Rules. Competitors must indicate the specific style on the entry form. Each entry will be judged based on the declared style." For the 'odd beer' category, you should specifically exclude beers that fit any 'known' style. These beers should be judged on drinkability and/or creativity. It is important to clearly state the purpose and judging criteria of this category and explain what types of beers are acceptable. You could also ask entrants to justify their entry by explaining what is odd about their beer, and provide this information to the judges. Since there is no effective way to establish objective criteria for this category, embrace subjectivity. If you have celebrity judges, this could be a style for them if they are adventurous. An example: "ODD BEERS: This category is provided for beers that fit no common style. Entrants are required to describe what is unusual about their beer on the entry form. For instance; "red pepper lambic", "triticale stout", or "marijuana barleywine". Entries will be judged for drinkability and creativity, according to the judges' personal tastes. These beers will be judged by Larry, Darryl, & Darryl from the Rather Silly Brewing Co." ----- Chuck Cox - SynchroSystems - chuck at synchro.com "All the other nations are drinking Ray Charles beer, and we are drinking Barry Manilow." - Dave Barry -------------------------------------- From JudgeNet Wed Jan 8 13:05:49 1992 Subject: Re: Generic Category The idea of a generic category is both good and bad. It's good because there are a lot of interesting beers out there that don't fit well in a category, and it seems a shame that these beers are not allowed to compete. It's bad because there are no style definitions and therefore no real way to judge the beer except things like "I really liked it" or "It's got extra GOLD in it". (Does everybody hate that commercial as much as I do?) I think I may have the start of an answer to this problem: Maybe the generic category should include more information from the brewer himself, such as what kind of beer he was shooting for (e.g. a "wheatwine" or a "dark American wheat beer"), or even just a description of the beer he was shooting for (e.g. a low-gravity, stout-colored, burnt-grain tasting, highly hopped lager). This information could be given to the judges (it would need to be retyped for secrecy) and they could use the brewer's own "style" as a basis for judging. I realize that there may be problems with judging a beer in this manner, that's why I said it was the start of an answer. Why don't we juggle this around a bit and see if we can come up with something that will work --- I think that having a generic category is a good idea if we can figure out how to make it feasible. Have at it... Michael L. Hall mlh at cygnus.ta52.lanl.gov or hall at lanl.gov -------------------------------------- From JudgeNet Wed Jan 8 13:40:40 1992 Subject: Re: Generic Category Hmm Russ, my personal opinion is that a generic category is a bad idea. It invites people (say like Jack Schmidling) who are too lazy or inexperienced to know what style a beer they've brewed, and which really does fit a style, to just enter it there. I think having such a category is asking for trouble as it will likely get swamped. Then what do you do when you have 60 "Generic" beers and few in other categories. That would make judging a real problem. What I've seen done is to have "specialty" categories which are catch all type of category, but at least define some common characteristic (like fruit, herbs, etc...). That might be safer to have a few specialty categories covering a reasonable range, than a single "generic" categoiry. - JaH -------------------------------------- From JudgeNet Wed Jan 8 15:11:25 1992 Subject: Re: Generic Category > It invites people (say like Jack Schmidling) who are too lazy or > inexperienced to know what style a beer they've brewed, and which really > does fit a style, to just enter it there. OK, I can't keep quiet any more... 8') I'll be a bit more diplomatic, and ask why it is that any and everything should be legal fodder for competition? There are specific categories because of traditional (mostly) methods of, and results from, brewing. Competition should be a challenge to see how well one can approach the "masters" with our own methods and scale. This philosophy doesn't necessarily have to coincide with good taste, though it may (for real beer afficionados, I suppose it almost always does). Just because one can brew a great tasting concoction - why else would one compete if it doesn't fit an acknowledged style - doesn't mean that one should be able to compete with it. A favorite old saying comes to mind: "Even a blind pig finds an acorn once in a while." If you brew to your own drummer, and like the results, by all means keep it up, and more power to you. If you wish to compete, brew something that can be well defined, which can as a result be judged fairly and objectively, and which is characteristic of something "real". BTW - in my mind, one sign of an accomplished brewer is the ability to reproduce a brew. To me, that would tend to indicate an understanding and control over ones' equipment, ingredients, and method. I think those characteristics are important, though many don't. I believe that encouraging competition using random successes is to some degree encouraging a loosening of standards, but that is purely my (anal) opinion. Cheers...Gary -------------------------------------- From JudgeNet Wed Jan 8 16:21:49 1992 Subject: Re: Re- Generic Category Reply to: RE>Re: Generic Category One of the reasons for this category is to include all those home brewers who are not into styles and who pride themselves in making "homebrew" for its own sake. Afterall that is one of the freedoms of brewing your own beer. Many of these concoctions are not simply herb or fruit specialty brews. We do have a specialty category at present and we do identify the "style" for the judges, ie Passion Fruit Lager or Rosemary Porter. As far as listing the specific unique ingredients for the judges goes, this could work for and against. Depending on how weird those ingredients might be, a judge could be prejudiced against some item which he/she feels shouldn't be included in a brew (Gummi Bears, anyone?). On the other hand, if someone were to use peanuts in a brew, I'd want to know because I am deathly allergic to them and would suffer greatly if I were to consume it. At the moment we are thinking of requesting the OG of the brew so we can serve them to the judges in a specific order. RW... -------------------------------------- From JudgeNet Wed Jan 8 16:32:49 1992 Subject: Re: Generic Categorys Generic categories...an interesting idea which I'm sure has been given much thought. Just a few thoughts on the issue. Jay's post: } I think having such a category is asking for trouble as it will likely get } swamped. Then what do you do when you have 60 "Generic" beers and few } in other categories. That would make judging a real problem. } } What I've seen done is to have "specialty" categories which are catch all } type of category, but at least define some common characteristic (like fruit, } herbs, etc...). That might be safer to have a few specialty categories covering } a reasonable range, than a single "generic" categoiry. As you noted Jay, these categories already exist to Some extent in the herb, fruit, and specialty categories. So since these aren't judged against a specific rigorous style, I think it is unfair to prevent brewers from competing with non-traditional beers, just because they don't have fruit, herbs, or spices in them. On the other hand, I judged the 'Odd Beers' category at the NYCHG competition, and I can definitely appreciate that there are problems. At NYC, the odd category contained everything which didn't have enough entries to have it's own category. This meant we had to judge American Light vs. Altbier vs. Fruit beer vs. Rauchbier. We were lucky because it turned out there were really only 4 quality beers out of the lot, so the job wasn't too tough, but it could have been a bad situation. I think the biggest problem we ran into with this type of category is the lack of information which actually reached the judges. Well run competitions usually use computerized labels, and there is limited space on these labels for information from the brewer. In addition, many entries invariably come in without sufficient information for the judges (e.g Not knowing what fruit is in a fruit beer, what spices in a Xmas beer). Therefore I think non-traditional categories should have additional space because the brewer needs to convey more information to the judges than in the case of traditional styles. What if someone wanted to enter a California Common Beer with honey, cinnamon, nutmeg, ginger, oranges and vanilla in it? I doubt there are many computer labels which could contain all that info, and without it, the judges would surely be confused. As you may have guessed I favor have a category like this, but it could only be done correctly in a very well run contest, where all pertinent information reached the judges (and beers were penalized if sufficient info was not provided). ON another front though... Perhaps, as was a problem in Manchester, determining the Best of Show should only consider traditional styles (for those who don't know, the two final beers in the national competition were a Munich Dunkel, and a Specialty Beer, the Munich Dunkel was chosen in the end because it was brewed to a more rigorous style). Chuck can tell us how good the Specialty Beer was...and the Dunkel for that matter. I'm not sure how I feel about this one. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc3.bitnet -------------------------------------- From JudgeNet Wed Jan 8 16:37:08 1992 Subject: re: Generic Category Russ, I don't have any answers but here are some additional questions: 1) Would it be acceptable for a "generic beer" to be awarded a "best of show" award? Would the other entrants feel that is fair? perhaps the generic beer competition is kept separate. 2) Is it even worth giving a score to a beer entered as "generic" when the entire judging process would be subjective? How about a pass/fail sort of rating (where pass = "worthy experiment" and fail = "botched process/idea/recipe/ingrediantes/etc...") 3) What would be the PURPOSE of a generic beer category? Is it to help out the novice brewer by, in essence, giving them access to trained palates/noses? Is it to make the competitions less rigid? Or is it to operate as a "testing ground" for new categories (the Olympic games has trial sports)? 4) What should be the judging rules for a well-made but ill-conceived beer? For example, if someone enters the best-possible Road-Kill Stout does that beer get a high or low score? 5) Some care must be taken to avoid any "ringers" from entering. For example, if someone thinks they have a good but not very good beer that would be middle-of-the-pack for a category competition, it doesn't seem fair for it be entered and do well as a generic beer. Someone else - sorry forgot who - said that an "originality" criterium be used. Okay but beware of someone purposefully making weird beer -instead of good beer- to win originality points. 6) Note that by establishing a generic beer group that the long-term effect _may_ be that the current beer judging categories will use stricter and stricter interpretations of the beer styles because judges might be more critical of beers that are slightly out-of- category. (This change may be good or bad). 7) What sort of beers are expected? If it truly is wide-open, then, as you mentioned, the score sheet has to reflect that. All of the standard categories could remain but might be weighted differently. There should be more space for comments. Maybe I can give you an opinion or two: About the concern that heavy beers would somehow be automatically favored by a judge over a lover O.G. beer -- I've judged fruit beers that ran the range of lightly flavored and bodied beers to heavily flavored beers. I don't believe there was any general bias. I think most judges are accustomed to somebody handing them a mystery beer for tasting. Perhaps the generic beers could be judged in this vein. In short, I like the idea since it would help out novices get accustomed to both the idea of competing and provide feedback earlier in their beer-making career. Besides, it sounds fun! -------------------------------------- From JudgeNet Wed Jan 8 17:27:10 1992 Subject: Re: Re- Generic Category Reply to: RE>Re: Generic Category >I'll be a bit more diplomatic, and ask why it is that any and everything >should be legal fodder for competition?... Gary; The reasoning is thus: Evaluating the ability of the brewer to brew a clean well made beer does not require the beer to match a specific style. Why not provide a vehicle for those brewers who only want feedback on their skills at brewing rather than clouding the issue with comments about where the beer is or isn't in keeping with a style which may well be a result of the recipe followed, which may not have originated with the entrant. Perhaps this sort of evaluation might be better done as a separate function or comp instead of mixing it with a style oriented comp. However, I doubt that we will see too many "generic only" competitions in the near future. To add the category to an existing comp is to test the water, and to take advantage of an event which is already organized. My understanding is that the first aim of homebrew competitions is to provide feedback to the brewer with the awards being second. Granted we all like to be a winner, but the majority of entries are not and they deserve the best feedback we can give. That's why we continue to hone judging skills and fine tune the comps. Your discussion tends to rule out the creation of new styles by pioneering brewers. Who's to say that what we have today is all there is? And I'm not advocating the abolition of the current style oriented comp. I'm just interested in getting the free thinkers and independents involved in things. Besides this is a hobby which is supposed to be fun. I'm for encouraging any who wish to participate. I completely agree with your comment "one sign of an accomplished brewer is the ability to reproduce a brew". I think it is an excellent way to evaluate abilities. That's what allows the pros to continue to stay in business. How do we work that into a competition atmosphere? This discussion is going better than I'd hoped. Thanks for the feedback and keep it up! RW... -------------------------------------- From JudgeNet Wed Jan 8 17:28:33 1992 Subject: Generic Category To some extent, I think a generic category would be a nightmare to administer, but I think Chuck Cox's analysis is as close as I'd like to get. On what Chuck calls "Other Styles": I have some sympathy toward this notion as I've seen otherwise excellent beers miss out because there was no category for them. On the other hand, I've deliberately left some categories out of the Oregon State Fair competition, not just because there was likely a dearth of entries, but because I had no confidence I'd be able to find qualified judges. (Insert long and boring diatribe about the inappropriateness of categories for which few judges have any real experience.) I see the same problem here, with people entering steinbier or something made from sorghum. I mean, how many steinbiers have you ever tasted? (Jeff's answer: 0) Many of the "Odd Beers" simply end up in our specialty category (we will have to separate out fruit beers this year; they overwhelmed the specialty category). These are "judged for drinkability and creativity, according to the judges' personal tastes," as Chuck suggests. I see no problem with providing a separate/different form for one of these categories. I do, however, think Jay Hersh has a very good point, or rather several: you will get swamped and likely by the arfs of the world; people who take the shotgun approach to brewing and to competitions will dump all their disasters on the category hoping one will strike the judges' fancy. Don't EVER ask me to judge a category like this. -------------------------------------- From JudgeNet Wed Jan 8 22:41:31 1992 Subject: Re: Generic Category >I think I may have the start of an answer to this problem: Maybe >the generic category should include more information from the >brewer himself, such as what kind of beer he was shooting for >(e.g. a "wheatwine" or a "dark American wheat beer"), or even just >a description of the beer he was shooting for (e.g. a low-gravity, >stout-colored, burnt-grain tasting, highly hopped lager). This >information could be given to the judges (it would need to be retyped >for secrecy) and they could use the brewer's own "style" as a basis >for judging. Yes, but hwat stops the brewer from defining the "target" after the fact (ie once he's brewed and tasted the beer he defines the target) then he always makes the beer he was supposedly shooting for. Won't work. I'm not sure if Gary Mason was agreeing with me or disagreeing with me. I wasn't endorsing that any and every beer should have a place in a competition, quite the opposite. My views are pretty much in line with his. I do feel however that a lot of brewers house brews do fit a style, they just don't know what it is. It seems to me that the folks Russ describes, those who just make "homebrew" more often than not fall into this group, and accomodating those who do brew to a style but just don't know it is a dangerous precedent. Odd Ales and Specialty beers have some ingredient/ingredients that set them apart from the traditional styles, and deserve their own places, but these categories are usually difficult to judge, and should be used judiciously for these reasons. I for one am wary of some of these new styles such as Texas Brown Ale, and California Common beer that seem to have cropped up lately. To date both the AHA, and Eckhardt define style by citing commercial examples. If you can't name at least one commercial reference (and I'd prefer two or more) then how can you define a style that the judges will have a common reference point to use when judging?? While I understand and appreciate the homebrewers drive to innovate and create (and I've tasted some damn awesome beers by Ron Paige one of the most innovative homebrewers on this coast) the nature of judging a beer is to have some standard against which to judge, and you just can't do that if you don't define at least some base level criterion (why I think some of these Odd Ale and/or Specialty Beer Categories are problematic). Of course another approach is to forget categories all together and rely on several panels of judges selecting the best beers in a tiered format where the top beers from each panel are then assembled, and judged again until the number of beers is reduced to the top 3 finishers. What criteria would be used here?? Solely the subjective tastes of the judges as to whether they liked it or not, without any hints or specialty ingredient info, or anything like that provided. Yes there will be a wide range in judge preferences, but assuming you have good judges who won't choose off flavor beers you would pick a good beer as the winner, but the winner would reflect the apples vs oranges prejudices of judges on the panels at all steps of the way. Is this unfair?? Well in this sytem no, since the goal is to pick a beer which the judges think is best in a criteria free format, and subjectivity plays an inherent part. - JaH ------------------------------------------------------------------------ Hopfen und Malz, Gott erhalts -------------------------------------- From JudgeNet Wed Jan 8 22:51:11 1992 Subject: Re: Re- Generic Category >Evaluating the ability of the brewer to brew a clean well made beer does not >require the beer to match a specific style I don't agree. There are flavor profiles that are characteristic for one category that are off flavors for another. Cloviness in wheat beers is a perfect example, the sourness of Lambics another. If you found these flavors in a Pale Ale you'd say it was a bad beer. > Why not provide a vehicle for >those brewers who only want feedback on their skills at brewing rather than >clouding the issue with comments about where the beer is or isn't in keeping >with a style which may well be a result of the recipe followed, which may not >have originated with the entrant. But this too is part of a brewer's skills. Witness Mike Fertsch who has brewed many a beer to a recipe for a certain style, only to recognize that the result was better suited to a different style, and subsequently won the category he entered instead of the category the recipe said. Blind allegiance to a label on a piece of paper shows lack of skill (especially since who knows if the person who did the labeling did so at the time of concoting the recipe or after actually tasting the result). >Perhaps this sort of evaluation might be better done as a separate function or comp instead of mixing it with a style oriented comp That makes more sense, though you still have the problem of style specific flavors and determing whether they're good/bad as I mention above. >My understanding is that the first aim of homebrew competitions is to provide >feedback to the brewer with the awards being second. An aim, I would not say that this aim is first, nor last, but a critical one which unfortunately is sometimes contradictory to the other aim of choosing the best beers (which requires some standard of reference as I mentioned in a previous message). >Your discussion tends to rule out the creation of new styles by pioneering >brewers. Who's to say that what we have today is all there is? No one, but IMHO style is by definition a matter of acceptance among the commercial brewing establishment, and inseparably linked the consuming public. The styles that exist today didn't arise in a vacuum in someone's kitchen. They exist because while someone was inspired to brew them, they gained an acceptance that made them last. Experimentation is fine, but I don't think it's the duty of competition organizers to accomodate every experiment that any brewer dreams up, no matter how fine it may be. If it is something lasting then it will establish itself in the propagation of it's popularity, otherwise let it be enjoyed for the unique experiment it is without there being a need to confer on it title. > I'm just >interested in getting the free thinkers and independents involved in things. For an entry fee of course :-), my experience has it that free thinkers and independents like being that way and will for the most part stay that way and feel no need to participate or have some honor bestowed on them for their efforts, they're independent enough to enjoy their efforts solely for their own sake. If you are bent on providing such a forum though I would strongly recommend dissociating it from a style category since I think it will undermine a style oriented competition. >I completely agree with your comment "one sign of an accomplished brewer is the >ability to reproduce a brew".How >do we work that into a competition atmosphere? Out here in beantown the Worts have held several "clone" competitions where a specific brand is the target. No reason this couldn't be done in a competition format, sort of like brewing to style, but even more specific. JaH -------------------------------------- From JudgeNet Wed Jan 8 22:54:57 1992 Subject: Re: Re- Generic Category I agree completely with the points raised about the goals of competitions, and particularly with the idea of a separate "competition" for feedback on generic brews. In fact, I think that idea might lure more beginners into submitting, therefore exposing themselves to expert evaluation and constructive comment. Without the stigma of facing off against the heavies, one might be more inclined to face the public in "competition". Furthermore, it might be better for the "normal" competitions by lowering the number of entries to be judged, thereby allowing the judges to concentrate on a tighter set of standards for the duration of the competition. I did not mean to have my personal conservatism creep in to the point of restricting others. I just happen to be one of those whose challenges lie along the more traditional lines. Cheers...Gary -------------------------------------- From JudgeNet Wed Jan 8 23:01:03 1992 Subject: re: Generic Category > Perhaps, as was a problem in Manchester, determining the Best > of Show should only consider traditional styles (for those who don't > know, the two final beers in the national competition were a Munich > Dunkel, and a Specialty Beer, the Munich Dunkel was chosen in the end > because it was brewed to a more rigorous style). Chuck can tell us how > good the Specialty Beer was...and the Dunkel for that matter. I'm not > sure how I feel about this one. > > Tom Strasser Both beers were excellent. We needed a tie-breaker, so we picked the Dunkel because it was the more difficult style to brew. > 1) Would it be acceptable for a "generic beer" to be awarded a > "best of show" award? Would the other entrants feel that is fair? > perhaps the generic beer competition is kept separate. > > 2) Is it even worth giving a score to a beer entered as "generic" > when the entire judging process would be subjective? How about > a pass/fail sort of rating (where pass = "worthy experiment" > and fail = "botched process/idea/recipe/ingrediantes/etc...") > > Bob Devine Judging, especially BOS will always be subjective, that's unavoidable. Anybody who thinks their beer will receive a perfectly objective evaluation is dreaming, regardless of category. Since subjectivity is unavoidable, the important thing is to be honest to the competitors about it. > Don't EVER ask me to judge a category like this. > > Jeff Frane No, ask me instead. Seriously, don't knock it 'til you've tried it. I've had the fortune (and occasional misfortune) of judging several odd beer categories. As others have pointed out, this is where the state of the art is. Sure, there are some dogs, but there are some killers too. Since many competitions have catch-all categories, I think all experienced judges should be able and willing to judge them. ----- Chuck Cox - SynchroSystems - chuck at synchro.com "All the other nations are drinking Ray Charles beer, and we are drinking Barry Manilow." - Dave Barry -------------------------------------- From JudgeNet Thu Jan 9 11:06:56 1992 Subject: Generic brewing... Jay - I didn't mean to confuse you (us). I guess my points are: 1. Competition to class (or other rigidly defined characteristics) as tier one. 2. A separate, and valuable, exercise for "teaching through judging" as tier two. 3. All brew is good brew if you like it. Just don't try to fit the square pegs into the round holes for the sake of doing it - it might just degrade the whole shebang. It certainly would seem to make the judge's job harder. Cheers...Gary -------------------------------------- From JudgeNet Thu Jan 9 15:06:57 1992 Subject: Re: Re- Re- Generic Category Reply to: RE>Re: Re- Generic Category >>I completely agree with your comment "one sign of an accomplished >>brewer is the ability to reproduce a brew".How >>do we work that into a competition atmosphere? > >Out here in beantown the Worts have held several "clone" competitions >where a specific brand is the target. No reason this couldn't be done in a >competition format, sort of like brewing to style, but even more specific. Well, that's not what I meant by "reproduce". I was talking about a brewer being able to consistantly reproduce a good, specific beer without variation in end result from batch to batch. Just like the pros have to do with their packaged product. This being a reflection of one's technical ability as a brewer. Along the line of the Worts' "clone comps" is the AHA "Anchor Steam look-a-like". There are problems here as well, in that the "original" has to travel and be stored and may be mishandled in the process. What a brewer in Maine considers a good clone (and in fact matches what he gets from his local source) may not come close to what the brewery considers correct. But this is straying from my original question. The more I think about this issue, the more I begin to agree that this isn't a category for competition. My reasons for including it are, for the most part, to provide an evaluation of the beer to the brewer. In a competition this will get done, however it may not be reasonable for these entries to "compete" for a place against each other. A compromise might be to have a stand alone competition with entries defined by color and OG and award metals for reaching a certain level of scoring. This would eliminate the "place" awards and simply recognize the merit of the brews. RW... -------------------------------------- From JudgeNet Thu Jan 9 19:26:15 1992 Subject: Re: Re- Re- Generic Category > A compromise might be to have a stand alone competition with entries > defined by color and OG and award metals for reaching a certain level of > scoring. This would eliminate the "place" awards and simply recognize the > merit of the brews. That's actually a very good idea, especially if you're trying to encourage folks who don't presently compete, and give them feedback on their skills. Of course this probably has nothing in it for the judges in way of experience points since it probably isn't a sanctionable event, but I'm sure you'll still find plenty of judges who are interested in helping out for the sake of goodwill and a chance to taste what's out there (not everyone is Rich Gleason, notorious point-monger... :-) - JaH -------------------------------------- From JudgeNet Fri Jan 10 13:22:01 1992 Subject: Generic category Greetings, I've been following the discussion of the generic category with some interest, and only have about 2-cents worth to add. Seems to me that with the large number of categories already defined by the AHA there really isn't a pressing need to add more. Sure it would be nice to bring novice brewers into the competition scene, but most beginners are probaly more concerned with working the kinks out of their procedures than with entering competitions. And with all the clubs around the country most any novice should be able to get evaluations of and advice about their brews for free if they take time to search out their local club. I don't remember just who it was who made the point, but it also seems unlikely to me that a generic beer would be in the running for best of show given the lack of a standard against which to evaluate it; that being the case, there doesn't seem to be a real need for such a category. [nonarchival comments deleted - MLH] Ooogy wawa, Dr. John -------------------------------------- From JudgeNet Mon Jan 13 15:13:12 1992 Subject: Serial vs. Parallel Judging Techniques Fellow judges, I realize that we just exhausted one topic and perhaps it's too soon for another, but one that IMHO is both interesting and important has cropped up. Six members of my club got together last night to compare two different judging techniques. One was the method currently used in homebrew compe- titions, consisting of judging 1 beer at a time, and I'll call this the "serial" method. The other consisted of having an entire flight (6 beers) in front of you at once, enabling you to do a side-by-side comparison, and I'll call this the "parallel" method. The parallel method is how wines are judged in competition, and the person who came up with the idea for this tasting (Chris Stamp) is a professional winemaker as well as an amateur beermaker. We judged standard North American lagers...Coors, Bud, Labatt's, Genessee, and two Meister Braus. Chris wanted to compare very similar beers of a style that's fairly "well-defined". I wanted to query the Judgenet readership: (1) why is the serial method used now for homebrew judging, historically- speaking? (2) what are some of the advantages/disadvantages you see to either of these techniques? (3) have any of you ever tried the parallel method in a competition or in a less-formal tasting? (4) would the AHA and/or HWBTA object if a sanctioned competition utilized the parallel approach? Obviously, I have some opinions on the matter based on last night's experi- ence, but want to hold off to avoid biasing the question I'm posing. I appreciate your input, STEVE -- =============================================================================== Stephen Russell Graduate Student, Department of Materials Science and Engineering Internet: srussell at snoopy.msc.cornell.edu work: 607-255-4648 Bitnet: srussell at crnlmsc3 home: 607-273-7306 =============================================================================== -------------------------------------- From JudgeNet Mon Jan 13 15:22:21 1992 Subject: who's who Here's the current subscription list, please send corrections/updates to judge-request at synchro.com. Here's the breakdown of the membership: 27 apprentice 9 recognized 7 certified 6 national 1 honorary 50 total ## Beer Judge Mailing List uunet!bandy at aplcen.apl.jhu.edu Mike Bandy - apprentice uunet!baughmankr at conrad.appstate.edu Kinney Baughman - certified uunet!dub at cornella.cit.cornell.edu Dwight Beebe - recognized uunet!peterb at cs.cmu.edu Pete Berger - apprentice uunet!embreed at sfovmic1.vnet.ibm.com Emily Breed - apprentice uunet!Lcarter at claven.idbsu.edu Loren Carter - certified uunet!jnsmc at alaska.bitnet Scott M Clark - apprentice chuck Chuck Cox - national uunet!bwc at icd.ab.com Barry Cunningham - apprentice uunet!jdecarlo at mitre.org John DeCarlo - apprentice uunet!tdenny at rigel.cs.pdx.edu Tom Denny - apprentice uunet!devine at cookie.enet.dec.com Bob Devine - apprentice fertsch Mike Fertsch - national fitz Jim Fitzgerald - national uunet!sequent!techbook!gummitch Jeff Frane - certified uunet!76702.764 at compuserve.com Robin Garr - apprentice uunet!jason at gibson.sde.hp.com Jason Goldman - apprentice semantic!judge-in Bob Gorman - recognized uunet!hall at buffa.enet.dec.com Dan Hall - apprentice uunet!mlh at cygnus.ta52.lanl.gov Michael L. Hall - certified uunet!Henchal at Detrick-emh1.army.mil Erik A. Henchal - apprentice uunet!hersh at expo.lcs.mit.edu Jay Hersh - national uunet!att!drutx!homer Jim Homer - honorary uunet!jeorg at chs.com Jeorg Houck - apprentice uunet!hopduvel!brewmaster at linac.fnal.gov John L. Isenhour - certified uunet!jelj at cornella.cit.cornell.edu John E. Lenz - recognized uunet!malodah at pbmoss.pacbell.com Martin Lodahl - certified uunet!tedm at hp-pcd.cv.hp.com Ted Manahan - recognized uunet!mason at habs11.enet.dec.com Gary Mason - apprentice uunet!rob at maccs.dcss.mcmaster.ca Robert J. McDonald - apprentice uunet!tmitchel at bbn.com Tom Mitchell - apprentice #uunet!MECE1XG at Jetson.UH.EDU Purna Murthy - apprentice uunet!bmyers at garnet.berkeley.edu Brian Myers - apprentice uunet!rcm at col.hp.com Rick Myers - apprentice uunet!reeves at essdp1.lanl.gov Geoff Reeves - apprentice uunet!resch at craycos.com Dave Resch - recognized uunet!darrylri at microsoft.com Darryl Richman - national uunet!roody at whzguy.shr3.dec.com Greg Roody - apprentice uunet!ross at mscf.med.upenn.edu Andy Ross - recognized uunet!srussell at snoopy.msc.cornell.edu Stephen Russell - recognized uunet!chris at asylum.gsfc.nasa.gov Chris Shenton - apprentice uunet!wslack!wrs at gozer.mv.com Bill Slack - recognized uunet!strasser at raj5.tn.cornell.edu Tom Strasser - recognized uunet!stroud%gaia at polaroid.com Steve Stroud - national uunet!gak at Corp.Sun.COM Richard Stueven - apprentice uunet!kswanson at nwu.edu Kurt Swanson - apprentice uunet!gdtms at garlic.lcs.mit.edu Greg Troxel - apprentice uunet!swh at ll.mit.edu Sarah White - apprentice uunet!Rad_Equipment at RadMac1.ucsf.edu Russ Wigglesworth - certified uunet!zulauf at orbit.colorado.edu Mike Zulauf - apprentice ## ----- Chuck Cox - SynchroSystems - chuck at synchro.com "All the other nations are drinking Ray Charles beer, and we are drinking Barry Manilow." - Dave Barry -------------------------------------- From JudgeNet Mon Jan 13 16:06:51 1992 Subject: Re: Generic Beer Category The Boulder homebrew club had a similar category in out last competition. We called it the Just Beer category. Judges were asked to judge based on their personal preference, indicating category if a beer did fit. The intent was to have a relatively large number of judges to compensate for any strange personal preferences. Sort of throwing out the highest and lowest scores. There were only 3 entries, all from experienced brewers. This was a disappointment, since we wanted to attract new brewers to the competition. We have heard that a competition in Florida had 60 entries for this type of category. Jim Homer att!drutx!homer -------------------------------------- From JudgeNet Mon Jan 13 23:11:20 1992 Subject: Re: Serial vs. Parallel Judging Techniques > > Fellow judges, > > I realize that we just exhausted one topic and perhaps it's too soon for > another, but one that IMHO is both interesting and important has cropped up. > > Six members of my club got together last night to compare two different > judging techniques. One was the method currently used in homebrew compe- > titions, consisting of judging 1 beer at a time, and I'll call this > the "serial" method. The other consisted of having an entire flight (6 beers) > in front of you at once, enabling you to do a side-by-side comparison, and > I'll call this the "parallel" method. The parallel method is how wines > are judged in competition, and the person who came up with the idea for this > tasting (Chris Stamp) is a professional winemaker as well as an amateur > beermaker. > > We judged standard North American lagers...Coors, Bud, Labatt's, Genessee, > and two Meister Braus. Chris wanted to compare very similar beers of a > style that's fairly "well-defined". > > I wanted to query the Judgenet readership: > > (1) why is the serial method used now for homebrew judging, historically- > speaking? > (2) what are some of the advantages/disadvantages you see to either of > these techniques? > (3) have any of you ever tried the parallel method in a competition or > in a less-formal tasting? > (4) would the AHA and/or HWBTA object if a sanctioned competition utilized > the parallel approach? > > Obviously, I have some opinions on the matter based on last night's experi- > ence, but want to hold off to avoid biasing the question I'm posing. > > I appreciate your input, > > STEVE > > -- > On an informal basis, or in a very small competition, there's no reason why a parallell competition wouldn't work just fine. The greatest problem is that it doesn't work well when you have to judge more than a few beers at a time (a given panel would probably need to judge more than twice this many beers at most events). Then, too, this method works for entirely subjective comparisons but I think it short changes the brewer expecting good, solid feedback. The "serial" approach requires the judge to evaluate each beer separately and more or less objectively. The form is designed not only to allow the judge to come to a reasoned result, but to provide concrete responses to the brewer. Telling someone that X's beer is better than yours isn't much help. -------------------------------------- From JudgeNet Mon Jan 13 23:21:20 1992 Subject: Re: Serial vs. Parallel Judging Techniques Fellow IBUer, Steve Russell asks: >(1) why is the serial method used now for homebrew judging, historically- > speaking? I'm surprised to hear that it is! >(2) what are some of the advantages/disadvantages you see to either of > these techniques? >(3) have any of you ever tried the parallel method in a competition or > in a less-formal tasting? >(4) would the AHA and/or HWBTA object if a sanctioned competition utilized > the parallel approach? While it's true that at every competition I've ever judged in used the serial method to present the beers to the table, I've always "forced" my team to leave a sample of each beer behind in his/her glass so we have the option of going back and tasting beers that had similar scores--essentially the parallel approach--at the end of the round. I know some people like to drink a beer. Score it. Go on to the next and let the numbers rule at the evenings end but I STRONGLY disagree with this approach, for several reasons, the most important being that it takes me two or three beers to get warmed up and into the mood of the competition. Not many of us judge so often that we can just sit down, evaluate a beer and be right on the mark the first time. Maybe my old basketball days influence me too much in this regard but I wouldn't think of walking onto the court for a game without running lay-ups and nailing about 10 from the free-shot line! Uppermost in my mind when it comes to judging a class is deciding what the top three beers are for that category. Close behind that, of course, is feedback to the brewer on his efforts, but when ribbons and bragging rights are on the line, it's important that a judges panel agrees that numbers 1, 2, and 3 are clearly 1, 2, and 3. I don't think you can do this if you toss each beer as you drink it. You're not being fair to the brewer if you just let "the numbers" rule. We all know how subjective judging is to begin with and I feel very uncomfortable with that kind of approach. Another point has to do with the fact that each judging event is a learning experience. If I'm judging beer with a brewer whose palate I respect, I'm all ears ( or is that all-buds!). If I learn something on beer number 5 that I didn't apply in my evaluation of beer number 1, then I'd like to have the latitude to go back and adjust my score. I don't think I could do that properly if beer number 1 is already in the dump bucket. So I guess there's no doubt I'm in favor of the parallel approach. I thought everyone judged that way!! :-) Have at it, you guys. Kinney Baughman | Beer is my business and baughmankr at conrad.appstate.edu | I'm late for work. -------------------------------------- From JudgeNet Tue Jan 14 14:07:10 1992 Subject: Re: Serial vs. Parallel Judging Techniques Kinney sez: > ... it takes me two or > three beers to get warmed up and into the mood of the competition. Not many of > us judge so often that we can just sit down, evaluate a beer and be right on the > mark the first time. Maybe my old basketball days influence me too much in this > regard but I wouldn't think of walking onto the court for a game without > running lay-ups and nailing about 10 from the free-shot line! Couldn't agree more. I fell into the classic trap judging barleywines at 10AM (truly the Breakfast of Champions!): the first one I tasted was just marvelous; virtually everything I could want in a barleywine, so I gave it a 17 (on a 20-point scale). The next was better, and the one after that better yet ... What saved the day was that among the three judges we accounted for somewhat less than half the bottle, which was immediately recapped (albeit by thumb) and returned to the cooler. We all kept track of our scores. At the end of the flight we asked for several to be brought back for a second look, including the ones with the highest scores and any we'd had second thoughts about. Sure, the sediment had been stirred up and the "first rapture" of esters was gone, but the possibility of a retaste was there. > ... We all know how subjective judging is to begin with ... Amen. I've been on a couple of panels with judges who were apparently unable to tolerate perceptual differences. Anyone scoring a beer differently from the score they'd given it was ipso facto wrong, and subject to attack. It's very difficult to be altogether open to the sensory qualities of the next beer in the flight when you're considering busting another judge's beak. > Another point has to do with the fact that each judging event is a learning > experience. If I'm judging beer with a brewer whose palate I respect, I'm > all ears ( or is that all-buds!). If I learn something on beer number 5 that I > didn't apply in my evaluation of beer number 1, then I'd like to have the > latitude to go back and adjust my score. I don't think I could do that > properly if beer number 1 is already in the dump bucket. > > So I guess there's no doubt I'm in favor of the parallel approach. My preference is the hybrid "series-parallel" method outlined above. A serial pass doing initial evaluations (it's important to me what a beer's qualities are the moment it's poured, and a few minutes later), with a second look at the best, the close, and the judge's mistakes. -------------------------------------- From JudgeNet Tue Jan 14 22:11:19 1992 Subject: More on the serial/parallel question RE: the serial vs parallel judging discussion, Jeff Frane sez: >[...] The >greatest problem is that it doesn't work well when you have to >judge more than a few beers at a time (a given panel would probably >need to judge more than twice this many beers at most events). The problem of having to judge too many beers at one time was brought up when we did our comparison. We did 6; some said the ideal number would be more like 4 or 5. >Then, too, this method works for entirely subjective comparisons >but I think it short changes the brewer expecting good, solid >feedback. The "serial" approach requires the judge to evaluate each >beer separately and more or less objectively. The form is designed not >only to allow the judge to come to a reasoned result, but to provide >concrete responses to the brewer. Telling someone that X's beer is >better than yours isn't much help. I agree wholeheartedly with your last sentence, which is why I think that subjective judging in and of itself is not enough. But I also don't think that subjective judging is valueless. How else do you decide, of two flawless, in-style brews, which is better? The parallel approach may indeed improve the 'subjective' part of judging, but that does not mean that it'll hurt the 'objective' part. Perhaps what you fear, then, Jeff, is that the subjective part would be given a higher 'weighting' under the parallel approach. As long as the objective part of the judging meets the purpose of providing feedback to the brewer, I contend that this doesn't really matter. Problems I _do_ envision: --as beers sit around, they lose conditioning (however, this means that the parallel approach will favor more well-conditioned beers) --they will also warm up (logistics...put the fridge down a notch further) --and aroma will be lost (again, the more well-conditioned beers will be less likely to get docked points here) --judges said they felt 'pressured' this way, even though they also favored this method (solution: break down into smaller subflights?) Among the advantages: --Enhanced accuracy in determining ranking. (a) Flaws. say you judge porter and you decide that beer #1 is too "roasty", so you dock it 3 points. Then you come to beer #8 which is also too roasty. How many points do you dock it? If you know just how roasty it is relative to beer #1, you'll be more fair (meaning, consistent) in your judging. (b) Drinkability. The subjective part is worth 10 pts out of 50 and often determines the winner(s) in a category. Relative drinkability would be assessed more accurately. --Unfairness due to palate fatigue is minimized. Beers judged at the end are judged less accurately due to 'palate fatigue'; this would to a great degree be eliminated by a parallel tasting where you can go back to beer #1 and reevaluate it anytime. Which should be encouraged in a parallel tasting. The combined serial-parallel approach seems ok, but the obvious problems are: --differences in the time since they were poured will affect aroma and conditioning and are difficult to take into account --and the temperature will be variable, too (unless you wait for all to equilibrate to room temperature), which will skew both the intensity of and balance among the various flavors Hopefully the first time around you'll get an idea about aroma, conditioning, etc., and the second (and third, etc.) time you taste you'll hone your score and ranking to enhance accuracy. Looks to me like there are now four techniques: (1) "Straight Serial" (taste one at a time, then discard) (2) "Second-guess Serial" (taste one at a time, but there is some around if you want to retaste) (3) "Small Group Parallel" (taste 3-5 beers at a time; some is saved if you want to retaste, probably) (4) "Full Group Parallel" (taste an entire flight side by side, regardless of number) I would like to hear from Jim Homer; I'm sure the AHA has considered this question. Surely the HWBTA has...I bet amateur winemakers mimic professional winemakers in their judging techniques. How do the GABF and Britain's GBBF assess commercial beers, anyhow? IBU ERGO SUM, STEVE "Ask two judges a question, and you'll receive three opinions." -- =============================================================================== Stephen Russell Graduate Student, Department of Materials Science and Engineering Internet: srussell at snoopy.msc.cornell.edu work: 607-255-4648 Bitnet: srussell at crnlmsc3 home: 607-273-7306 =============================================================================== -------------------------------------- From JudgeNet Wed Jan 15 22:11:17 1992 Subject: Re: More on the serial/parallel question > > RE: the serial vs parallel judging discussion, Jeff Frane sez: > > >[...] The > >greatest problem is that it doesn't work well when you have to > >judge more than a few beers at a time (a given panel would probably > >need to judge more than twice this many beers at most events). > > The problem of having to judge too many beers at one time was brought up > when we did our comparison. We did 6; some said the ideal number would > be more like 4 or 5. > My problem as a supervisor is that I simply cannot gather enough judges to keep the panels down to 4 or 5 beers. Then, too, most of the categories run far larger--in some cases, more than I can have a single panel judge. Using a point-based "objective" system allows there to be some rationale in weighing results from Brown Ale Panel A and Panel B. In point of fact, with the State Fair, we ended up having a third panel re-tasting the top three beers from each panel to arrive at the winning beers. In one case where we intended to do this, the opinion of all the judges was that one panel had no winning beers and the other had two extraordinarily good ones, so we didn't need to use the same selection process. > >Then, too, this method works for entirely subjective comparisons > >but I think it short changes the brewer expecting good, solid > >feedback. The "serial" approach requires the judge to evaluate each > >beer separately and more or less objectively. The form is designed not > >only to allow the judge to come to a reasoned result, but to provide > >concrete responses to the brewer. Telling someone that X's beer is > >better than yours isn't much help. > > I agree wholeheartedly with your last sentence, which is why I think that > subjective judging in and of itself is not enough. But I also don't think > that subjective judging is valueless. How else do you decide, of two > flawless, in-style brews, which is better? > > The parallel approach may indeed improve the 'subjective' part of judging, but I hope I never suggested that "subjective judging is valueless." I am the last one to suggest that truly objective judging even exists! Obviously, as you say, one needs to make a choice between two flawless--or more likely--equally flawed beers. > that does not mean that it'll hurt the 'objective' part. Perhaps what you > fear, then, Jeff, is that the subjective part would be given a higher > 'weighting' under the parallel approach. As long as the objective part of > the judging meets the purpose of providing feedback to the brewer, I contend > that this doesn't really matter. > In general, I would say that Steve's advantages and disadvantages are reasonably thought out. I have to say that, once again as an organizer, the logistics of comparison tastings are horrendous. Keeping track of the beers is difficult (especially for some judges) and the greatest problem for us is having enough glassware. Last year I had about 30 judges tasting 150 beers (yes, I know: that's too many beers/judge, but what can you do?) With three judges on a panel, that's a lot of glasses! > I would like to hear from Jim Homer; I'm sure the AHA has considered this > question. Surely the HWBTA has...I bet amateur winemakers mimic professional > winemakers in their judging techniques. How do the GABF and Britain's > GBBF assess commercial beers, anyhow? > > > IBU ERGO SUM, > > Yes. Let's hear from Jim. > > > > -------------------------------------- From JudgeNet Wed Jan 15 22:18:17 1992 Subject: Serial/parallel Judging I favor the Serial-Parallel method as described by Kinney. This is the procedure that the experienced judges I know, and learned from use. The AHA policy from the 1991 NHC Manual for Judging Procedures is: 5. When all judges are ready, bottles will be opened one at a time and only as they are judged so that each individual entry is judged completely before opening and continuing to the next entry. In the event of ties and close scoring, it is appropriate to re-evaluate previously judged entries, but the fairness must be applied to all entries (time, temperature, light, exposure, etc.). In the GABF Professional Panel Blind Tasting, a modifyed serial-parallel method is used. Beers are brought into the judges as quickly as they can be poured, judges make brief notes on each beer. The judges then determine the results using their notes and sampling the beers in front of them. The serial part is quicker than we are used to with homebrew judging. This because score sheets are not used, only simple feedback forms. Jim Homer att!drutx!homer -------------------------------------- From JudgeNet Thu Jan 16 22:12:32 1992 Subject: Serial/parallel Judging } From: uunet!att!drutx!homer at uunet.UU.NET } } I favor the Serial-Parallel method as described by Kinney. } } This is the procedure that the experienced judges I know, } and learned from use. } } The AHA policy from the 1991 NHC Manual for Judging Procedures is: } } 5. When all judges are ready, bottles will be opened one at } a time and only as they are judged so that each individual } entry is judged completely before opening and continuing } to the next entry. In the event of ties and close scoring, } it is appropriate to re-evaluate previously judged entries, } but the fairness must be applied to all entries (time, } temperature, light, exposure, etc.). I had a question as to the clarity of this statement. I think that additional parallel judging is only one interpretation of this statement. From my limited experience, only one judge I've ever judged with, or seen judge, has used this technique. All other judges I've seen have done their "re-evaluation" by just talking about the best beers after they had finished the flight. Auf ein neues, Tom Strasser...strasser at raj5.tn.cornell.edu...strasser at crnlmsc3.bitnet -------------------------------------- From JudgeNet Fri Jan 17 01:12:22 1992 Subject: Re: Serial/parallel Judging Jim Homer quotes: >The AHA policy from the 1991 NHC Manual for Judging Procedures is: > 5. When all judges are ready, bottles will be opened one at > a time and only as they are judged so that each individual > entry is judged completely before opening and continuing > to the next entry. In the event of ties and close scoring, > it is appropriate to re-evaluate previously judged entries, > but the fairness must be applied to all entries (time, > temperature, light, exposure, etc.). Just a data point here. The first time I judged in a competition, I had the good fortune to judge with Pat Baker. This was about 5 years ago and the BJCP was in its infancy, if not its embryonic stage. I've been thinking about the hybrid serial/parallel approach that I advocated in an earlier post and I finally realized that was the way Pat suggested we conduct the tasting that night. Subconsciously (and now consciously) I have since adopted that method as my own judging style. I can recall my reasoning: I'm just a po' ol' country boy and here's a guy whose palate and know-how I wouldn't even think of criticizing. If it's good enough for him, it's good enough for me! :-) --------- Don't want to interrupt this thread but I have a question to pose to this august forum (even if it IS January!). I've observed two distinct approaches to evaluating a beer. I'll call the first approach the "bottom up" method. With the "bottom up" method, the judge goes step by step through the judging form, logging in points as he/she goes, summing up after the last category, and going on to the next beer. The "top down" method assigns the point total first, then goes back and fills in each category until the points match the total. I'll withold my opinion for the moment. I'd be interested in seeing a discussion of the method employed by the various members of the list. If you prefer not to discuss the issue publicly but would still like to register your preference, email me at: baughmankr at conrad.appstate.edu and I'll report the results to the list. A simple "bottom up" or "top down" as the body of the message will suffice. Just make the SUBJECT LINE, "My preference". Thanks. Kinney Baughman | Beer is my business and baughmankr at conrad.appstate.edu | I'm late for work. Hey! You East Coasters! Was it cold enough for you today? 0 degrees F. at the old homestead this morning. BRRRRRRRRRRRRR!!!!!!! Give me a BEEEEEEEEEEEER!!!!!!! -------------------------------------- From JudgeNet Fri Jan 17 12:23:01 1992 Subject: National "additional" round Subject: National "additional" round Time:8:42 AM Date:1/16/92 Several people have expressed concerns to me as to the merits of the elimination process currently in use for the AHA Nationals. The obvious problems which arose in Boulder as a result of the size of the competition have been addressed by the total "regionalization" of the first round. This allows more judges to be involved and ought to provide each entry with a panel of three judges to determine its worthyness as a second round contender (by score). There are still some problems with the selection process. For the smaller categories, those with fewer than 15 entries per site, entrants can be assured that their wares will be examined by one panel of judges and the top beers will pass on to the second round. This may also apply somewhat to those categories which have as many as 30 entries. The same panel of judges could judge 2 consecutive flights over one day. While this does not compare all the entries at one session, at least the same palates are awarding the points to all of them. The real problem comes with the larger categories. The system is relying on several panels of judges, possibly as many as 5, working through maybe 60 entries. We rely on the resulting scores "alone" to determine those which will pass to the second round. The "maximum 7 point spread" requirement tries to address this to some degree, however it does not take into account scoring trend variations between panels. Would it be too much to add one more step to the judging and make up a flight from the resulting top scoring beers, either off the top of the total category, or from each panel of judges (both have problems), and have another panel select those which will pass on to the second round from this? Speaking as the West Coast Registrar, I don't think it would be too hard. Based on my projections for the number of entries to expect this year, I'd say we are only talking about 4 or 5 categories at each site, if we allow the middle size categories to be judged in consecutive flights by the same panel. This would require that (2) bottles be sent for the first round. That is the biggest drawback that I can see, needing more storage space and making more work for the crew unpacking the entries. This may or may not be resolved and/or introduced this year, but I thought this would be a good place to talk about it. Several of the people involved in this competition have access to this forum either directly or indirectly. If you have access to one of these folks, please pass this along. So let's talk it up! RW... Russ Wigglesworth CI$: 72300,61 |~~| UCSF Medical Center Internet: Rad Equipment at RadMac1.ucsf.edu |HB|\ Dept. of Radiology, Rm. C-324 Voice: 415-476-3668 / 474-8126 (H) |__|/ San Francisco, CA 94143-0628 -------------------------------------- From JudgeNet Fri Jan 17 21:11:20 1992 Subject: Re: National "additional" round Russ: I think the idea of an extra step in judging is excellent, if you can land the judges. As I mentioned in another context, we followed a similar procedure for a couple of categories at last year's Oregon State Fair, and I expect to this year. I think it's much more fair than relying on the mathematical results of two very different panels. As a brewer entering a competition, I would want my beer judged by as many mouths as possible. There isn't, I think, any reason for the "final" judges to fill out judging forms: a strictly subjective, side-by-side assessment would be entirely appropriate, I think. On the other hand, it would be good if there was *some* form that was sent to the brewers in explanation. -------------------------------------- From JudgeNet Fri Jan 17 21:16:23 1992 Subject: Re: Serial vs. Parallel Judging Techniques >(1) why is the serial method used now for homebrew judging, historically- speaking? Large flights (10-15, AHA defines the max at 15 beers per judge but this is often ignored) and the importance of tasting at consistent temperatures. also the need to rinse and re-use glassware or otherwise purchase large amounts of plastic cups (ie cost). >(3) have any of you ever tried the parallel method in a competition or in a less-formal tasting? Yes it is good for less formal settings. also it works great for when you're narrowing down your finalists. I and others I have judged with typically use serial as a first pass to weed out the weaklings, then do a parallel on the best in flight. > It's very difficult to be >altogether open to the sensory qualities of the next beer in the >flight when you're considering busting another judge's beak. Hmm, Martin perhaps you haven't exercised sufficient tact. I've almost always been able to "convince" a judge whose perceptions were way out of line (I think 5 points either way is OK, but more than that starts to demonstrate a lack of knowledge of the category between a more and a less experienced judge) by educating them on what the proper characteristics of that category were. They usually end up saying "oh yeah I hadn't looked at it that way" Of course this assumes I'm judging something I'm familiar with, which is after all something to be expected of a concentious judge. I save the experimentation for tasting sessions, not competitions. - Jay -------------------------------------- From JudgeNet Fri Jan 17 22:11:28 1992 Subject: Re: "Top-Down" vs. "Bottom(s)-Up" Kinney B. introduced a fascinating new thread: > I've observed two distinct approaches to evaluating a beer. I'll call > the first approach the "bottom up" method. > > With the "bottom up" method, the judge goes step by step through the judging > form, logging in points as he/she goes, summing up after the last category, and > going on to the next beer. > > The "top down" method assigns the point total first, then goes back and > fills in each category until the points match the total. I'll go on record, here and now, as being a "bottom up" judge. Sure, I develop a general idea of what range the overall total should be, and occasionally find myself reviewing my individual attribute scores if the final total is too high, but the general model is "bottom up". When I've tried the "top down" method I end up having to "fudge" some attributes a bit too violently. Take, for example, the category most judging systems have to describe color, clarity, and conditioning. There's not a lot of room, there. You can move a point or two in either direction over color in some classes (though a stout either is black, or it isn't). A beer is either cloudy or it's not, so there's not much you can do there, if you have to. And if it distributed itself across the table (and judges) when opened, it may leave a poor impression (especially if you're wet the rest of the day, and smell like Al Capone's warehouse while driving home), but can you really mark the score down for more than the conditioning points (assuming no other flaws)? A related problem is the occasional beer whose parts are individually fine, but just don't add up to a good beer on the whole. I met one in the Nationals last year that I felt was a pretty extreme case of this, though the other members of the panel disagreed. My "top down" score would have been pretty poor, but I couldn't have justified that total in my "bottom up" evaluations. Which leads to my final supposition, and then I'll shut up (honest!): I think that the "bottom up" method will usually produce higher total scores than the "top down" approach. Y'all disagree? = Martin A. Lodahl Pacific*Bell Systems Analyst = = malodah at pbmoss.Pacbell.COM Sacramento, CA 916.972.4821 = = If it's good for ancient Druids, runnin' nekkid through the wuids, = = Drinkin' strange fermented fluids, it's good enough for me! 8-) = -------------------------------------- From JudgeNet Sat Jan 18 19:04:37 1992 Subject: Re: "Top-Down" vs. "Bottom(s)-Up" I use a hybrid approach to this (;-). I judge by marking comments down as I go through the beer without marking any points. In fact, I write up all of my comments before I think about points. Naturally, I have a good idea of where the beer ought to fall at this point, but I go over each judging area and score the beer, lettting the final score add up as it may. Experience in doing this allows me to not overscore a beer. As Martin points out, bottom-up scoring does tend to yield higher scores, but an awareness of this, and an understanding of how to hold down scores can keep this in check. In particular, things like "Other Fermentation Charactistics", "Balance" and "Aftertaste" tend to get small scores from me unless a beer is really a top contender. And, of course, this is also what Overall is all about. --Darryl Richman -------------------------------------- From JudgeNet Mon Jan 20 16:07:48 1992 Subject: Serial/Parallel judging experience Judges, Last weekend at the Boston Brewoff I attempted to "put to the test" the serial/parallel compromise as discussed on judgenet last week. My category was Brown Ales, and we had 14 beers. No stewards; I acted as steward and did the pouring for myself and the other two judges. I only poured 2-2.5 oz in each cup, and then put the cap back on the bottle and set it aside. If the beer was a 'contender', i.e. one of the better beers in the flight, we saved it for the end. Each beer was tasted and scored serially. At the end, there was one beer that was head-and-shoulders above the rest. We saved it anyhow just to drink (it ended up taking best-of-show). Four others had average scores of 28, 29, 30 and 31. So, we took what was left of them and re-tasted them, side-by-side, ranked them in order from 1 to 4, and the top two got their 2nd and 3rd place ribbons. I felt much more comfortable doing it that way as opposed to blindly sticking to the averages to determine ranking. We didn't pay much attention to aroma or appearance at that point, just aimed for a straight-up flavor comparison. The order we ended up in was indeed slightly different than what straight averaging would have yielded. Had the differences in scores been greater, I would have had no compunction about dropping the final, parallel tasting, but a one-point difference doesn't mean much (the first place beer was a 41, btw). I felt that the final parallel tasting enhanced fairness. If we had had a steward, it may have been more difficult to do things this way. We might have had problems due to non-conformity with other tables. I will try to push for this method in future competitions, however. If anyone else who judged the Brewoff did anything different this past weekend due to previous discussion on judgenet, please chime in! Cheers, STEVE =============================================================================== Stephen Russell Graduate Student, Department of Materials Science and Engineering Internet: srussell at snoopy.msc.cornell.edu work: 607-255-4648 Bitnet: srussell at crnlmsc3 home: 607-273-7306 =============================================================================== -------------------------------------- From JudgeNet Tue Jan 21 10:17:23 1992 Subject: Re: Serial/Parallel judging experience I was an apprentice at the Boston Brew-Off this past weekend, and thought I'd relate my experience. We had 14 beers to judge, 1 certified judge and 2 apprentices. We ended up with three beers that were head and shoulders above the rest (and a couple that were knees and feet below the rest...). After scoring each beer, we came back to the top three, retasted each, and made sure the scores (43, 42, 41) reflected our 1st, 2nd, and 3rd place finishers. So I guess we followed the serial/parallel method too. It seemed to work well, and though I've never judged before, it seems like a must to come back and reexamine the top contenders side by side. Since this was my first time judging beers, I was able to better understand the discussions on this list like serial-parallel and top-down/bottom-up. I found myself jotting down some notes, deciding where the beer should fall score wise, and then going back putting a score in each slot, and seeing where they add up. If the total was close to my initial impression I'd leave it. If not, I'd take a second look to see if I could adjust any of the scores to better reflect the initial score I decided on. The long and short of it is that judging beer is difficult. Some of the categories on the score sheet don't leave alot of room for deduction (color/clarity), so others are more susceptible to deduction (flavor, overall impression). I suppose I'll be learning alot more in the future about judging, and I'm sure it will help me interpret the scores of my own beers. Does anyone know if apprentices can sit in on the judging for nationals? Cheers, -tom -------------------------------------- From JudgeNet Tue Jan 21 10:56:31 1992 Subject: Re: Serial/Parallel judging experience I judged BOS along with Steve Russell at last weekend's Boston Brewoff. Some comments: I think that a best of show demonstrates some of the strong and weak points about parallel judging. Namely, one can quickly pick out the good beers, so the process tends to go faster, but the poorer ones don't get a lot of attention because they are quickly discarded. I have always been a proponent of the serial-first, parallel-second judging style. I think that everyone who has entered a competition deserves a well-filled out score sheet with constructive comments. This is especially important for the weaker entries, since they are the ones who really need the feedback. It is my experience though, that the score for any given beer can be influenced to some degree by the order in which it was served, i.e. which beers were served just before it. Therefore, unless, the ranking is REALLY obvious, I like going back and re-tasting (parallel) the top scoring beers as has been discussed in this forum recently. In a competition this means that the organizer needs to limit the number of judges in each panel. Personally, if the judges are required to fill out score sheets, I would NEVER allow more than three judges on a panel unless a second bottle was available for a "parallel" judging. If only one bottle is available, use 3 judges maximum, use the "pour half/recap" method initially, then reopen the bottle if retasting to determine rank is necessary. There may be an issue with cloudiness here, but all beers will have been treated equally. If two bottles are available, then this doesn't apply and, in fact, five or six judges could be placed on each panel! The AHA National Finals (i.e. second round) have been notoriously bad about this (there is only one bottle per entry). Up until last year, basically anybody who showed up was allowed to judge, which made for some really *large* panels, often 5 or 6 judges. I remember judging the bocks in 1990 . The first place beer was obvious (Hi, Darryl!), but there was disagreement over 2nd and 3rd. Unfortunately, we had 5 judges and had drained each bottle on the first pour. We had no way of going back and comparing beers even though we really needed to. Last year the AHA had their act more together and limited the number of judges. I helped judge weizens along with two members of TRASH-US. Three judges - fifteen beers - and we were able to do a good job of serial-first/parallel-second judging. I hope this method is used in Milwaukee. Steve Stroud -------------------------------------- From JudgeNet Tue Jan 28 15:11:50 1992 Subject: Them forms Hey how about form order??? I never judge in the order. I always do appearance first, it only takes a second or two to evaluate (hey they're basically free points...), then I do aroma, then body, then flavor, and finally overall impression. My reasoning is that before you smell you look, then you smell. Body is something that you sense right away in the first taste in your mouth, while flavor is more complicated and has more components so requires additional tastes and contemplation. Finally when all these have been accounted for an overall impression can be made... How about you?? How do others do this?? - JaH -------------------------------------- From JudgeNet Tue Jan 28 15:13:57 1992 Subject: AHA Nationals.. told you so... AHEM... not being one to pat myself on the back and say I told you so :-), but long time readers of the HBD will recall a discussion held several years ago caling for the creation of a tiered competition system where winners judged at a local or regional site would then be forwarded on to the National Round finals. At the time of the discussion the AHA competition was smaller, but there were also a lot fewer judges. I think back then ( late 87 or 88) the AHA just did not have confidence in anyone outside of Boulder to take these responsibilities. While much controversy raged around this issue (and some of us let our AHA memberships lapse over it...) time has vindicated those championing such a system. Now if the AHA would only actually let it's members have some real representation, say like having an elected Board of Advisors who actually had some authority to make policy over certain non-publishing issues (such as judging and competition policy) I'd be really happy. Still in all they have been a lot more responsive in the last few years accepting suggestions such as moving the conference around... - JaH ------------------------------------------------------------------------ Hopfen und Malz, Gott erhalts -------------------------------------- From JudgeNet Wed Jan 29 00:11:05 1992 Subject: RE: Them forms | I never judge in the order. I do pretty much follow the form order. | I always do appearance first, it only takes a second or two to evaluate (hey | they're basically free points...), then I do aroma, then body, then flavor, | and finally overall impression. I find that there can be a number of aromas that can disperse quickly, especially some of the sulphur based ones. I immediately dive into aroma to catch these things. (But I come back to aroma afterwards as well, to see how the initial aromas hold up.) | My reasoning is that before you smell you look, then you smell. You can look at the beer at any time; in fact, in judging retention, you might want to come back to the beer a bit later anyway. I leave taste for last because it gives the beer one last chance to gain a degree or two (most contests serve beer too cold). --Darryl Richman -------------------------------------- From JudgeNet Wed Jan 29 00:16:12 1992 Subject: Re: Them forms Jay Hersh (one of the few National-level judges) introduces a new thread: > Hey how about form order??? > I never judge in the order. > I always do appearance first, it only takes a second or two to evaluate (hey > they're basically free points...), then I do aroma, then body, then flavor, > and finally overall impression. First I get some sense of the conditioning from the opening of the bottle and the appearance of the pour. The first thing I do when I reach for the glass, though, is breath in the fumes, through both nose and mouth. That "first rapture" of a just-poured beer will often show you both the best and worst, IMHO. Then I evaluate the color and the clarity, then back to the smell. At last, a taste -- I agree with Jay about judging body -- and the overall impression will quickly form from there. Then I retaste to get the flavor specifics. Talking about all this is making me thirsty ... = Martin A. Lodahl Pacific*Bell Systems Analyst = = malodah at pbmoss.Pacbell.COM Sacramento, CA 916.972.4821 = = If it's good for ancient Druids, runnin' nekkid through the wuids, = = Drinkin' strange fermented fluids, it's good enough for me! 8-) = -------------------------------------- From JudgeNet Wed Jan 29 18:17:16 1992 Subject: Point systems Subject: Point systems Time:10:34 AM Date:1/29/92 There was a discussion of the 20 point vs. the 50 point scoring system at our club meeting last night. I'm curious as to what the rest of you use/prefer. My own opinion is that 20 points are plenty! The use of half-points allow the range to be doubled and offer plenty to cover all sorts of beer quality. RW... Russ Wigglesworth CI$: 72300,61 |~~| UCSF Medical Center Internet: Rad Equipment at RadMac1.ucsf.edu |HB|\ Dept. of Radiology, Rm. C-324 Voice: 415-476-3668 / 474-8126 (H) |__|/ San Francisco, CA 94143-0628 -------------------------------------- From JudgeNet Wed Jan 29 18:20:33 1992 Subject: Re: Them forms > > Jay Hersh (one of the few National-level judges) introduces a new > thread: > > > Hey how about form order??? > > I never judge in the order. > > I always do appearance first, it only takes a second or two to evaluate (hey > > they're basically free points...), then I do aroma, then body, then flavor, > > and finally overall impression. > > First I get some sense of the conditioning from the opening of the > bottle and the appearance of the pour. The first thing I do when I > reach for the glass, though, is breath in the fumes, through both > nose and mouth. That "first rapture" of a just-poured beer will > often show you both the best and worst, IMHO. Then I evaluate the > color and the clarity, then back to the smell. At last, a taste -- > I agree with Jay about judging body -- and the overall impression > will quickly form from there. Then I retaste to get the flavor > specifics. Talking about all this is making me thirsty ... > I'm with Martin. But... on the other hand, what difference does it make what order the forms are laid out? No matter how you prioritize things there, people will continue to judge in the order that's most comfortable and useful to them. My concern in the past was in re-ordering things so that proper weight was given to the various categories (previously, much too much attention was paid to appearance/color). Now that that's been sorted out, I find the form much more useful. --Jeff Frane -------------------------------------- From JudgeNet Thu Jan 30 03:15:17 1992 Subject: Re: Point systems (20 vs. 50) Russ Wigglesworth advanced another great topic: > There was a discussion of the 20 point vs. the 50 point scoring system at our > club meeting last night. I'm curious as to what the rest of you use/prefer. > > My own opinion is that 20 points are plenty! The use of half-points allow the > range to be doubled and offer plenty to cover all sorts of beer quality. I agree with Russ, but mostly because I simply don't have a 50-point palate. Hell, there are times I doubt I have a 20 point palate ... = Martin A. Lodahl Pacific*Bell Systems Analyst = = malodah at pbmoss.Pacbell.COM Sacramento, CA 916.972.4821 = = If it's good for ancient Druids, runnin' nekkid through the wuids, = = Drinkin' strange fermented fluids, it's good enough for me! 8-) = -------------------------------------- From JudgeNet Thu Jan 30 11:10:18 1992 Subject: Re: Point systems (20 vs. 50) Russ W. asks about using a 20 point vs a 50 point judging form. Russ, I'm in favor of wide point forms. If it were up to me, we'd have a 100 point form! When it was in use, I always found the 20 point scale too cramped. Great beers vs good beers vs OK beers were only separated by a few points. Wider ranges give the judge more latitude in scoring and allow easier ranking of the entries. Yes, you could use half points or quarter points or whatever to effectively double the range, but by doing that you've effectively agreed that the wider range is better! The most important argument for using the 50 point scale, though, is that it is used by all AHA/HWBTA approved competitions. Why would you want to train your club members to use a 20 point scale, then send them out to competitions (or the BJCP exam!) where a different form/scale is used? Our club always uses the AHA form for competitions. Steve Stroud -------------------------------------- From JudgeNet Thu Jan 30 11:22:58 1992 Subject: Re: Point systems >My own opinion is that 20 points are plenty! The use of half-points allow the >range to be doubled and offer plenty to cover all sorts of beer quality. Yeah but the math gets reall annoying having to add and divide with fractional points to get average scores... :-) JaH (no calculator on my waist) -------------------------------------- From JudgeNet Thu Jan 30 17:17:12 1992 Subject: Re: Re- Point systems (20 vs Reply to: RE>Re: Point systems (20 vs. 5 >>The most important argument for using the 50 point scale, >>though, is that it is used by all AHA/HWBTA approved >>competitions. Why would you want to train your club >>members to use a 20 point scale, then send them out >>to competitions (or the BJCP exam!) where a different >>form/scale is used? Our club always uses the AHA form >>for competitions. >>Steve Stroud Well this is exactly what prompted the discussion at the meeting the other night. The 20 point system was in use by our club (San Andreas Malts) prior to the AHA adopting the 50 point system. The club has always had a view of itself as a pioneer (it created the first state-wide competition in California in 1978) and also likes to continue the traditions it has established over the years. Consequenty there are a lot of judges in the Malts (as well as in the Bay Area) who were "born and raised" on the 20 point system. (There are also issues with the AHA involved here which are purely political and which I won't discuss here.) Myself included in that group, I find my judging mindset to be tuned to the 20 point system. Even admitting the value of the wider range by using half points, I find it easier to think in the 20 point range than the 50. This could simply be a result of lack of exposure to, or use of, the 50 point system. I am sure that I would adapt if it were selected as the universal norm. It is to this point that I asked what the rest of you use/prefer. If we are to "standardize" judging it makes sense that we all use the same forms, however, that isn't the case at the moment. I know why the Malts use the 20 pointer, I'm looking at what the rest of the country does. RW... -------------------------------------- From JudgeNet Thu Jan 30 17:25:41 1992 Subject: Re: Point systems > > Subject: Point systems Time:10:34 AM Date:1/29/92 > There was a discussion of the 20 point vs. the 50 point scoring system at our > club meeting last night. I'm curious as to what the rest of you use/prefer. > > My own opinion is that 20 points are plenty! The use of half-points allow the > range to be doubled and offer plenty to cover all sorts of beer quality. > > RW... > > Russ Wigglesworth CI$: 72300,61 > |~~| UCSF Medical Center Internet: Rad Equipment at RadMac1.ucsf.edu > |HB|\ Dept. of Radiology, Rm. C-324 Voice: 415-476-3668 / 474-8126 (H) > |__|/ San Francisco, CA 94143-0628 > I have this argument with Fred Eckhardt every time I have to use one of his forms. I simply don't believe 20 points is enough to cover gradations in beers. There's a big difference between a 30 point and a 40 point beer, in my mind, and no way to reflect it with a smaller range: yeah, yeah, I know I can use the decimal system but it just doesn't have the impact when the brewer gets it. As an administrator, I have to say that the 50 point system is a lot simpler to use when adding up points. --Jeff Frane > -------------------------------------- From JudgeNet Thu Jan 30 18:26:11 1992 Subject: Re: Point systems I thought I would like to add my two scents to the 20/50 point discussion: > As an administrator, I have to say that the 50 point system is a lot simpler > to use when adding up points. As someone who often gets roped into adding up scores and tallying winners, I find a lot of tie scores using the 50 point system: we might get two beers with average scores of 37.5 points. I think that using a compressed 20-point system would cause more numerical ties. A wider scoring system would result in fewer ties; I vote for a 100-point score. -- Mike Fertsch home: mikef at synchro.com synchro!mikef work: fertsch at adc1.adc.ray.com -------------------------------------- From JudgeNet Thu Jan 30 23:14:34 1992 Subject: Re: Point systems I was waiting for someone to suggest a 100 point system. I wouldn't even consider a 20 pt system for the same reasons that have been stated (the compression of scores problem). I think that a 100 pt system would be better than a 50 pt system because it's more familiar, in the sense of percentages. Did you ever have a professor in college that insisted on giving you a score like 35/60, or 15/20? I remember one instance where a professor gave out scores on a basis of *four* points. I got a 3 1/2! I wasn't sure whether to be happy or sad until I converted it to an 87.5% score. Of course, I don't expect it to change from 50, because that is the current AHA standard. And I'm not saying that I have that degree of accuracy in my judging. As Martin Lodahl said, sometimes I don't even have a 20 pt. palate. I just think that even though a 100 point system would have less meaning due to "just making up numbers" than a 20 point system, it would also have more meaning because it would be more familiar to the populace in general. Mike Hall mlh at cygnus.ta52.lanl.gov -------------------------------------- From JudgeNet Fri Jan 31 19:15:14 1992 Subject: Point systems My experience is that the AHA system encourages bunching of scores in the high 20s to low 30s. (Of course, that may just be me, but most of the judges I've worked with haven't had much problem with my scoring.) Reducing the range by more than half might create more ties than the 50 system, unless points are alloted in a different proportion than the AHA 50 point scheme. One question, though: if you use a 20 point system but award in half point increments, isn't that equivalent of using a 40 point system? I would see a lot less distinction between 40 and 50 than 20 and 50. --Darryl Richman -------------------------------------- From JudgeNet Fri Jan 31 19:17:38 1992 Subject: Re: Point systems >As someone who often gets roped into adding up scores and tallying winners, >I find a lot of tie scores using the 50 point system: we might get two beers >with average scores of 37.5 points. I think that using a compressed 20-point >system would cause more numerical ties. A wider scoring system would >result in fewer ties; I vote for a 100-point score. Ummm Mike, this is the AHA, you don't get to vote remember :-) - JaH -------------------------------------- From JudgeNet Fri Jan 31 23:08:52 1992 Subject: Re: Point systems I said once: >> I vote for a 100-point score. JaH cleverly responds: > Ummm Mike, this is the AHA, you don't get to vote remember :-) I should hold my tongue here, but I will snap back: I'm not sure what the AHA asks the regional organizers, or the Board, or Harris Faberman, but just how did the AHA decide on the current scorecard? A flash of inspiration in Boulder? BTW- I like the current 50-point scorecard. No critisism from me... -- Mike Fertsch home: mikef at synchro.com synchro!mikef --------------------------------------