To: judge-recipients at synchro.com (JudgeNet Recipients)
From: judge-owner at synchro.com (JudgeNet Administrator)
Reply-To: judge at synchro.com (JudgeNet)
Errors-To: judge-error at synchro.com
Precedence: bulk
Subject: JudgeNet Digest #1197 (Jan 16, 1996)
Date: 16 Jan 96 01:18:21 EST (Tue)
Message-Id: <9601160118.AA08764 at synchro.com>


JudgeNet Digest #1197				   Tue 16 Jan 1996


                               JudgeNet
			 The Beer Judge Digest

                 digest submissions: judge at synchro.com
          administrative requests: judge-request at synchro.com
    send cancellations & rank updates to the administrative address
	  messages sent to the wrong address will be ignored

        WWW Archives: http://www.umich.edu/~spencer/beer/judge

                 Editor: Chuck Cox <chuck at synchro.com>
             Archivist: Spencer Thomas <spencer at umich.edu>

      Publishers: SynchroSystems and the Riverside Garage & Brewery

            Anti-Prohibitionists may also be interested in 
                LiBeerty: The Libertarian Beer Digest
           Subscription info: libeerty-request at synchro.com

     For BJCP General Information contact: geninfo at bjcp.synchro.com


Contents:
  Re: Newsletter & Exams                   (Dennis Davison)
  Re: Subcategories                        ("Lee C. Bussy")
  Newsletter/Mailing                       (Greg Holton)
  Re: ENTRY GUIDELINES PER SUBCATEGORY     (Steve Dempsey)
  Re[2]: Newsletter & Exams                (Wolfe)
  Entries per category                     (STROUDS)
  Guidelines Part 13 (The End!)            ("Tim Dawson")

----------------------------------------------------------------------

Date: Mon, 15 Jan 1996 03:59:14 -0600
From: ddavison at earth.execpc.com (Dennis Davison)
Subject: Re: Newsletter & Exams


Ed Wolfe States:
>
>There is one thing, however, that I want to bring up as an issue.  Of
>course, it has to do with the Exam Format Changes article.  The article
>states that making people retake both sections of the exam "means we don't
>trust the validity and reliability of our exam."
>
>I believe that "trust" is not what we want when talking about validity and
>reliability of the exam--We should have "evidence" of its validity and
>reliability.  Unfortunately, I suspect that there is NO evidence of the
>test's validity and that there is only scanty evidence of its reliability.
>If there is, then I this information should be made public so that the
>members can evaluate it themselves.

Did you not mention to me when you recently re-took your exam that you did
not study for it as much as you did the first time you took it? This point
alone makes it difficult to get evidence of reliablility. I'm not singling
you out Ed, but I've regraded exams because the examinee felt that he should
have gotten a higher score, and moved up in rank after re-taking the exam.
The question at this point falls back onto how much did he study before the
exam, or did he feel that what he had learned at competitions over the past
year was enough for him to get another 5 points on the written, and another
6 points on the tasting.

This is not always the case. One thing that the BJCP should do is show
examples of a perfectly answered question. A perfectly commented score
sheet. This alone will be of great help to those that plan on retaking parts
of the exam.


>I propose that the BJCP, like any other certification program, should make
>an effort to evaluate the reliability, validity, and fairness of the BJCP
>exam and to make this information available to BJCP members.  This
>information will, in no way, jeapordize the security of the test, and it
>can be produced fairly inexpensively.  No other certification group that I
>know of requires an examination for membership but does not make this
>information available to its members.

This, though a big undertaking, may not lead to an accurate answer.
Curiculum in schools, as well as what in general should have been taught, is
easy enough to gain statitics on, but when you have a variable in that
people taking the BJCP exam are basically on their own to study the
material, I can't see where you will get consistant data. In other words,
when people are 8 years old they should know what 2+2 equals, yet, how can
we verify what judges have read when some may study for an hour and others
for three weeks.

Speaking of the exams that I re-graded because of formal protests, I was
looking to overturn the intial grade for these people. After reviewing the
exams, I couldn't. They just did not show the depth of knowledge required to
advance to the next level.

As far as statistics go:

                <60          60's        70's        80's         90's
Normal Range   0-15%        20-30%      40-50%      10-20%        0-5%

This has been the breakdown that exams have fallen into for the last ten
years. If an exam site has a higher or lower than normal curve, we do look
back at the exams to find out why, or if it was a grading problem on our
part. In gerneal, it's been lack of studing.


Dennis Davison                                       ddavison at earth.execpc.com

                           BJCC Midwest Region
                         Chairperson of the BJCP


------------------------------

Date: Mon, 15 Jan 1996 07:09:38 +0000
From: "Lee C. Bussy" <leeb at southwind.net>
Subject: Re: Subcategories

On 15 Jan 96 at 1:16, JudgeNet Administrator wrote:

> Do we really have to limit brewers from sending different examples
> of their chosen style into our competitions by this rule.   

We shouldn't have to but I think unfortunately we need to.

I have sat with a few of my fellow judges and tasted beers in the 
same category from the same brewer that were obviously the same 
recipe.  Not just close, not just a style that I had a passing 
aquaintance with.  The same beer in a style that I  know well.

Why did this brewer do this?  I can't comment authoritatively but I 
can surmise that his intentions were not honorable.  

This doesn't even begin to touch upon the subject of people entering 
the same beer in different subcategories because they either can't 
figure out what it is supposed to be or that they don't trust that 
the judges will recognize a beer that is representative of the style.
 
> Dan can comment ,  as could Steve Stroud on our local agreements and
> refresh my memory .....as I'm in a weakened state from heavy
> pharmaceuticals  but as to my intent , it's open season on the
> concept. Is there a good reason why we don't allow Jim Bob to enter
> the 6 different versions of his bitters recipe for evaluation? ? ?

As long as those bitters are in fact different.  It should however be 
up to the individual organizer based upon his regional makeup and 
past experience.

Without pointing fingers, we all know someone who has done this.  I 
think it is pure chickensh*t but it happens.  In a perfect world we 
wouldn't have to worry about it but we all know what kind of world we 
live in don't we?

- --

 -Lee C. Bussy/leeb at southwind.net

* I feel sorry for people who don't drink.  When they wake up,
* it's as good as they are going to feel all day.

------------------------------

Date: Mon, 15 Jan 1996 08:47:37 -0500 (EST)
From: greg at kgn.ibm.com (Greg Holton)
Subject: Newsletter/Mailing

I haven't seen my annual mailing yet, but I received my newsletter a few 
days ago and it lists my old rank on the mailing label.  I earned the points
to advance from recognized to certified 10 months ago ... this does not
inspire confidence.

 Greg Holton
 President, Hudson Valley Homebrewers, inc.

------------------------------

Date: Mon, 15 Jan 1996 08:44:54 -0800
From: Steve Dempsey <steved at ptdcs2.intel.com>
Subject: Re: ENTRY GUIDELINES PER SUBCATEGORY


In JudgeNet Digest #1196 BREWS said:

>-----------------------------
>Brewers may enter more than one beer per
>category, but only one beer per sub-category is allowed,
>----------------------------- End Original Text -----------------------------
>
>Do we really have to limit brewers from sending different examples of their
>chosen style into our competitions by this rule.   

>... Is there a good reason why we
>don't allow Jim Bob to enter the 6 different versions of his bitters recipe
>for evaluation? ? ?

As a competitor, I don't want Jim Bob to take home first, second, and
third place should all his beers receive top scores.  This problem can
be eliminated with a rule stating a brewer can receive only one prize
per (sub)category.  It makes things interesting though when you have
to make sure there are no ties for 4th place, etc. in case you have
to dig further down in the ranks to get 3 individuals.

As a judge, I don't want to sit through 6 variations on a theme by the
same brewer.  It's hard enough when someone enters the same beer in two
subcategories judged at the same table and everyone says "haven't we
tasted this beer already?"  Jim Bob should choose which of his 6 bitters
he thinks is best and enter that one.  If he doesn't know, he can bring
them to a club meeting and get some other opinions to help him decide.

 -Steve

------------------------------

Date: Mon, 15 Jan 96 13:51:33 CST
From: Wolfe at act.org
Subject: Re[2]: Newsletter & Exams

First of all, my original post was not derogatory.  I appreciate the work
that has been done for the BJCP by the new board.  It's a thankless job
much of the time.  My reason for raising the issue of exam reliability and
validity is that I think that it is a good way to improve the quality of
the exam (and, I can lend expertise to the effort).


Anyway, Denis replied:

.Did you not mention to me when you recently re-took your exam that you did
.not study for it as much as you did the first time you took it? This point
.alone makes it difficult to get evidence of reliablility.

There are multiple types of reliability.  You are mentioning only one of
them--stability over time (test-retest reliability).  There is also
comparability between graders (interrater reliability), stability between
sets of items (alternate form reliability), and stability within a test
(internal consistency)--none of them requiring the same person to take the
test more than once .  Of these four types of reliability, I think that
test-retest reliability is the least interesting and pertinent to
examinees.

He continued with:

.I'm not singling you out Ed, but I've regraded exams because the examinee
.felt that he should have gotten a higher score, and moved up in rank after
.re-taking the exam.  The question at this point falls back onto how much
.did he study before the exam, or did he feel that what he had learned at
.competitions over the past year was enough for him to get another 5 points
.on the written, and another 6 points on the tasting.

Just to make it clear, Denis is not saying that I have asked for a
regrade--I've never protested a BJCP exam score.  Although, if I did
complain, the exam grader has no alternative but to spend time regrading
the exam.  On the other hand, if you've established that the pool of items
are interchangeable (alternate forms reliability) that the graders are
assigning comparable grades (interrater reliability), and that the
certification standards are defensible, then the examinee's protest is
invalid.  The analyses that I referred to in my original post aim to answer
the question you raise--"how much did he study before the exam or ...
learned at competitions over the past year ..."

Denis continues with:

.One thing that the BJCP should do is show examples of a perfectly answered
.question. A perfectly commented score sheet. This alone will be of great
.help to those that plan on retaking parts of the exam.

I agree.  I think that this instructive role is an important one for the
examination program.

In response to my suggesting that the BJCP undertake studies of the
reliability and validity of its examination, Denis responded:

.This, though a big undertaking, may not lead to an accurate answer.
.Curiculum in schools, as well as what in general should have been taught,
.is easy enough to gain statitics on, but when you have a variable in that
.people taking the BJCP exam are basically on their own to study the
.material, I can't see where you will get consistant data. In other words,
.when people are 8 years old they should know what 2+2 equals, yet, how can
.we verify what judges have read when some may study for an hour and others
.for three weeks.

I think you're focusing on a different question than I'm suggesting, Denis.
You are correct.  If we set out to determine the relationship between
quality and quantity of examination studying and scores on the examination,
then we'll have a big mess on our hands.  It will be expensive, time
consuming, and will be difficult to assess the validity of our study.  What
I'm suggesting, however, is that we determine whether the scores are
reliable in ways other than test-retest reliability (although we can do
that too) as I suggested above and that we determine whether the items are
actually measuring something that we can label "judging
ability/skill/qualifications/etc."  Currently, I know of no evidence that
the test is valid or fair other than the testimonials of the people who
created it.  If that kind of evidence were the only kind cited by a testing
corporation or certifying agency, then that organization would find itself
innundated with law suits.

Denis finishes his post by stating:

.If an exam site has a higher or lower than normal curve, we do look back
.at the exams to find out why, or if it was a grading problem on our part.
.In gerneal, it's been lack of studing.

You don't know this.  That's exactly my point.  Your theory is likely to be
correct, but you have no grounds to make this statement unless you've done
the studies to verify:  1) that everyone in the pool of graders is
comparable (otherwise, an examinee may get a higher or lower grade
depending on who grades the exam) and 2) that all possible sets of items
are comparable (otherwise, an examinee may get a higher or lower score on
the exam depending on what set of items he/she gets).  To my knowledge,
both of these are risks in the current system.  I could be wrong, but isn't
it true that each examination site gets a different set of questions and
that the examinations from a site are sent to a specific group of graders
(who may or may not grade exams from the next examination site)?  If so,
and if there is no evidence that the pool of exam questions and the pool of
exam graders are comparable, then your theory is nothing more than a
theory.  You have no evidence one way or the other.

Again, I'm not protesting about the examination system--I'm only
suggesting ways that I think that the current system can be improved with
minimal effort and resources.  In fact, my proposal is to collect evidence
to support theories like the one Denis stated above.  That way, we can
avoid having to regrade exams (a waste of BJCP time and money) and
examinees can rest assured that the scores they receive are accurate and
fair.  My offer still stands--I volunteer my services to undertake such a
study.

Ed Wolfe
wolfe at act.org


------------------------------

Date: Mon, 15 Jan 1996 17:33:49 -0400 (EDT)
From: STROUDS at cliffy.polaroid.com
Subject: Entries per category

Brews -

I am not aware of any agreement that was ever made regarding this point in the 
NEHBOTY competitions.  If you have written documentation that says otherwise, 
please let me know.  My recollection is that at the Kingston meeting in 1994 it 
was suggested that the decision about # of entries/category would be decided by
each organizer, but the minutes from that meeting say nothing on that point. 

I don't believe that the matter was discussed at the Burlington 1995 meeting.  
Again, the written minutes don't reflect it.

In general I do not keep entry forms from past competitions, but a quick look 
through my files indicated that the GMM competition in both 1995 and 1994 only 
allowed one entry per subcategory per brewer.  The same was true for the 
Westport competition back in 1993 and the Maltose Falcons Mayfire Comp. in 
1992.  Why are you suddenly noticing the BHC?

The Deerfield competition has always allowed multiple entries for brewers.

My feeling is that it should be up to each individual organizer - why are you 
trying to tell other competitions what to do?  I feel that there are valid 
arguments on both sides and we will no doubt hear them in this forum in the 
coming days.  

Steve Stroud


PS - I wish that you had asked privately first about this issue - I doubt that 
many people on JudgeNet really care about what was or wasn't decided regarding 
NEHBOTY competitions.


I think that it _is_ valid to put a generic question out, asking people what 
they think about allowing multiple entries/brewer/subcategory (not category) in 
competitions.


------------------------------

Date: Tue, 16 Jan 1996 00:49:15 +500
From: "Tim Dawson" <strange-ti at mv.mv.com>
Subject: Guidelines Part 13 (The End!)

Hello Everyone,

Well here we are, the final posting of the guidelines.  We will be 
dealing with Wheat Beers.  But before I continue I would just like to take a few seconds to thank everyone that has helped out on th
is project.  I would especially like to thank Al Korzonas for all his 
help and I'd like to thank the members of Brew Free Or Die for all their help and support.  

My guidelines are not "finished", but this portion of them has come 
to an end.  I will continue to update them with new information and styles as I come across them.  You can always find an up-to-dat
e version on my web site at:
     http://www.mv.com/ipusers/strange/styles.html.

I will be writing to everybody that wanted a copy of the completed 
project soon.

OK, here we go!  Again, all comments should be E-Mailed directly to 
me at:
     tim at strange.mv.com

WHEAT BEERS

Weizenbier (or Weissbier) - Wheat beers of southern Germany.  Light 
to medium bodied, lightly hopped, yeasty, highly effervescent, and suggestive of cloves and bananas.  They are, above all, a summer
 beer.  At least 50% wheat malt.  Some cloudiness is acceptable in 
this style since a mash of wheat can add haze from protein.  Fermented as an ale by unique yeast strains.  Clove and slight banana c
haracter is required.  Vanilla, nutmeg, smoke and cinnamon-like 
phenolics are permissible.  Fruity/estery.  Mild sourness is OK.  Bitterness is low to unnoticeable.  Low hop flavor and aroma is OK.  
No diacetyl.  Light straw to amber.
Commercial examples: Paulaner, Hofbrauhaus, Schneider Weisse, Julius 
Echter Weizenbier, Edelweiss, Spaten Club-Weisse, Erdinger Kristall Weissbier, Oderdorfer Weiss.
O.G.: 1.045 - 1.055;  Alcohol: 4.5 - 5.5%;  IBU's: 8 - 15;  SRM: 3 - 
9.

Hefe-Weizen - Overall the profile of this beer is similar to Weizen.  
A good wheat beer mit Hefe has a complex flavor but does not have the grittiness of autolyzed yeast or wort protein.  Clove and s
light banana character is required.  Vanilla, nutmeg, smoke and 
cinnamon-like phenolics are permissible.  Fruity/estery.  Mild sourness is OK.  Bitterness is low to unnoticeable.  Low hop flavor and 
aroma is OK.  No diacetyl.  Light straw to amber.  This is a real ale 
style that is conditioned in the bottle or keg and will contain some yeast sediment.  Lager or ale yeast may be used to condition
 the beer.
Commercial examples: Pschorr Weizen, Wurzburger, Paulaner 
Hefe-Weizen, Prince Luitpold Hefe-Weissbier, Erdinger Mit Feiner Hefe-Weizen, Schneider Hefe-Weizen.
O.G.: 1.045 - 1.055;  Alcohol: 4.5 - 5.5%;  IBU's: 8 - 15;  SRM: 3 - 
9.

Dunkel Weizen - Dark version of Weizenbier and can be a bit stronger. 
The color is deep copper to full brown.  Slight chocolate-like or roasted maltiness may be evident. Medium bodied beer with an em
phasis of dark malt.  It usually has a little less of the 
characteristic clove-banana aromas.  The combination of wheaty tartness and the lusciousness of dark malts makes this style full of flavor an
d complexity.  Low diacetyl is OK.  Low hop flavor and aroma is OK.  
The classic examples are unfiltered.
Commercial examples: EKU, Hecker-Pschorr Dark Wheat, Oberdorfer 
Dunkelweizen, Erdinger Dunkel Weizen, Tucher Dunkles Hefe Weizen.
O.G.: 1.045 - 1.055;  Alcohol: 4.5 - 6%; IBU's: 10 - 15; SRM: 16 - 23

Weizenbock - Stronger and more robust than Dunkelweizen.  A medium- 
to full-bodied beer, it is made from 40-60% wheat, but the palate emphasis is on the high malt flavor.  Hop bitterness is low to un
detectable.  No hop flavor or aroma, but the clove and banana flavor 
and aroma are still evident.  Color can be from light amber to dark brown. Dark versions may have a mild roast malt flavor and aro
ma.  Alcoholic strength should be evident.  Low diacetyl is OK.
Commercial examples: Erdinger Pinkantus, Shneider Aventinius.
O.G.: 1.066 - 1.080;  Alcohol: 6.5 - 7.5%;  IBU's: 10 - 20; SRM: 7 - 
25

Berliner Weisse - This tart, refreshing, thirst-quenching beer can 
only be brewed in Berlin, Germany, although a few brewers in Northern Germany brew wheat beers in a similar style.  Often called the
 Champagne of beers.  Anywhere up to 75% malted wheat is used and 
results in a characteristic foamy large white head which tends to die quickly due to a lack of protein structure.  The ale-type yeast
 and up to 20% lactic combination produces a light body which is dry, 
tart, and sharp lactic sourness.  Very pale yellow color, effervescent, modest alcohol content, no bitterness, no hop flavor or a
roma and low fruity ester notes.  No diacetyl.  May be mixed with 
sweet syrups.
Commercial examples: Berliner Kindl Weisse, Schultheiss Berliner 
Weisse.
O.G.: 1.028 - 1.032;  Alcohol: 2.5 - 3.5%;  IBU's: 3 - 6;  SRM: 2 - 
4.

American Wheat Beer - A standard ale yeast is used.  Typically have 
light, crisp grain flavors and aromas characteristic of wheat.  The clovey aromas and flavors of Bavarian Weizenbiers are absent (a
nd inappropriate).  Low to medium fruitiness and esters.  Hop 
bitterness can be from medium to none.  Hop aroma and flavor  can be high or low.  The proportion of wheat is often greater than 50%.  Li
ght to medium body, pale straw to gold although light amber versions 
exist.  Low diacetyl is OK.  The use of lager yeast is OK.  No phenolic character.
Commercial examples: Wheat Hook, Samuel Adam's Summer Wheat, Oregon 
Original Hefeweizen, Catamount American Wheat.
O.G.: 1.030 - 1.050;  Alcohol: 3.5 - 5%;  IBU's: 5 - 30;  SRM: 2 - 8.

Wit or Belgian White Beer - The orange taste and sweet-dry spiciness 
of Belgian-style wheat beer sets it apart from other wheat beers.  This beer has a low to medium body and is brewed with up to 50%
 unmalted wheat, malted barley, and maybe oats.  The unmalted wheat 
gives Witbier more firmness of body and graininess then German Weizen beers.  It is stronger and maltier than Berlin Weiss but not 
as acidic.  Wit is tangy and sharply refreshing with hints of orange, 
honey, and even muscat.  They typically have a full yellow-white color and sport  very white heads.  Coriander seed, Curacao oran
ge peel, Hallertauer and/or Saaz may all be used.  Low to medium 
bitterness. Hop flavor and aroma from "noble-types" is desired.   A young example will have a fruity, quenching quality, but the bottl
e-conditioned varieties will mature over two to three months, during 
which time the hop flavor will recede and the orange flavors will be accented, possibly by some Madeira notes.  Dry.  Low diacetyl
 is OK.  Has low to medium fruity esters.  Bottle conditioned.
Commercial examples: Hoegaarden Witbier, Celis White, Steendonk, 
Blanche de Namur, Titje, Wieckse Witte, Spring Street 1444 Wit.
O.G.: 1.044 - 1.050;  Alcohol: 4.5 - 5%;  IBU's: 15 - 30;  SRM: 2 - 
4.

Graetzer - Graetzer beer is of low gravity and strongly hopped with 
"noble-type" hops.  It is made from 2/3 smoked, highly roasted wheat malt and 1/3 pale barley malt.  A single step infusion mash is
 usually used.  The flavor is very smoky and the style is very rare.
O.G.: 1030 - 1034;  Alcohol: 3 - 3.5%;  IBU's: 50;  SRM 17.

Grodzisk - Grodzisk is a specialty of Grodzisk, Poland, near Poznan.  
It is made with a significant proportion of malted wheat, smoked over oak.  The beer is top-fermented, perhaps with some wild yea
st influence, and bottled-conditioned.  It can be low in alcohol or 
of conventional strength.  It is an extremely pale golden beer, with a faint haze, a dense white head, and a surprisingly light bod
y.  It has a sourish, sappy, oaky aroma (like a box that had held 
smoked herring), and a smoky, very deep, crisp palate.  After a period of storage, it begins to develop a tart, quenching acidity.

Tim Dawson               "So many beers...
tim at strange.mv.com
Manchester, NH               So little time!!"
Brew Free or Die

------------------------------


End of JudgeNet Digest
************************