To: judge-recipients at synchro.com (JudgeNet Recipients)
From: judge-owner at synchro.com (JudgeNet Administrator)
Reply-To: judge at synchro.com (JudgeNet)
Errors-To: judge-error at synchro.com
Precedence: bulk
Subject: JudgeNet Digest #1204 (Jan 25, 1996)
Date: 25 Jan 96 01:19:54 EST (Thu)
Message-Id: <9601250119.AA23428 at synchro.com>


JudgeNet Digest #1204				   Thu 25 Jan 1996


                               JudgeNet
			 The Beer Judge Digest

                 digest submissions: judge at synchro.com
          administrative requests: judge-request at synchro.com
    send cancellations & rank updates to the administrative address
	  messages sent to the wrong address will be ignored

        WWW Archives: http://www.umich.edu/~spencer/beer/judge

                 Editor: Chuck Cox <chuck at synchro.com>
             Archivist: Spencer Thomas <spencer at umich.edu>

      Publishers: SynchroSystems and the Riverside Garage & Brewery

            Anti-Prohibitionists may also be interested in 
                LiBeerty: The Libertarian Beer Digest
           Subscription info: libeerty-request at synchro.com

     For BJCP General Information contact: geninfo at bjcp.synchro.com


Contents:
  Split batches and multiple entries       (BrewsMead)
  BJCP get's an F in judge training.       (John DeCarlo              )
  Re: Historical Braggot with Hops         (Spencer W Thomas)
  Re: BJCP get's an F in judge training.   (Spencer W Thomas)
  Re: Historical Braggot with Hops         (Fred Hardy)
  Re: JudgeNet Digest #1203 (Jan 24, 1996) (Greg Kushmerek)
  Comments on the grading rant             (Steve Russell)
  Food for Thought                         (bosko)
  Grading ranks                            (Scott Bickham)
  Comparisons of exam scores and letter grades (Scott Bickham)
  re:BJCP get's an F in judge training.    (drbeer)
  Alts/Judge Training                      (Algis R Korzonas)
  testing, and more                        (Craig Pepin)
  Jim Ellingson's comments on judge training..... (Jim Cave)
  judges with dietary restrictions         (Mark Taratoot)

----------------------------------------------------------------------

Date: Wed, 24 Jan 1996 04:59:56 -0800
From: BrewsMead at eworld.com
Subject: Split batches and multiple entries

 Craig Pepin touches directly on the reason I initially thought of for my
post to start this discussion. No it wasn't cuz I like to pick on the Worts ,
altho that is fun from time to time.

I myself am to be chastized if the cheating edge is when you vary the same
brew by yeasts or hops variation. There is a fundamental need for feedback
from others to verify the concepts put forth here as to the legimate, I feel
, need to test beer recipes via competitions.

If the reason you enter the beer is to close out the category ( prevent
others from winning) or merely win ribbons for self flaggelation, then I
suppose there is no help for you and shallowness has its own paybacks. You
must be a miserable SOB if this is your only reward in life and screwing with
funloving brewers is how you get off.

On the other hand , there are those of us who may need to get the feedback
from other judges in blind review, of beers that may be slightly different ,
where the subtlety is of paramount import to the fine lines of style,if you
care of such things. 

I like to win ribbons like the next guy , but the reason the beers get
entered and checks written is to have those scoresheets that indicate the
merits of stylistic comparison to our standard. 

I know whether the beer is good or not and I always drink it for the time
oriented variations to see change in the particular recipe but I feel that we
should leave open the subcategory entries total to allow for the scientific
pursuit of knowledge. 

There are those who say bring it to the meetings. Well no one likes to fill
out the forms as far as I know. I do it at competitions only because there's
a contract with the entrant and mail fraud requires it.

I'd rather not fill out the forms and drink the beer in flights of 4-6 to
determine the BOC and BOS by consensus. If its a competition , that's all
that matters  , who won and who lost. Since it's an evaluation as well as a
comp ,then we fill out the forms to give feedback .

I feel that I'm paying for beer evaluation by experts with better snouts &
palates than my own. So therefore I want what I'm paying for , evaluations of
my entries to style. If I'm in search of the perfect pale ale /IPA /APA the
base beer is very similar in grist competition(lay off me you perfectionists)
as well as mash style and can easily be modified by hops. 

At what point do you add hops for the best resultant bitterness / flavor
impact.How long do you boil for after that. If you are a retentive record
keeper like me and graph out all the timelines for this effect then you know
where I'm coming from.

You can boil the tar out of Chinook grown here at my house and with one
addition at 2-3 hours ,still get aroma. Only by trial and error do we improve
our perception and that is what I advocate, the experience of trying it for
yourself. Paying good money to be told your IPA sucks and is a 22 beer
provides essential feedback to my mind.   Way off meathead, back to the
board!!!

Rules are rules and we will live by this one as it applies to the Boston HBC
but I felt that the concept deserved our brief attention here to flesh out
the issues underlying it and the real reason we're in this .

Keep experimenting !             Brews

------------------------------

Date: Wed, 24 Jan 96 08:46:09 EST
From: John DeCarlo               <jdecarlo at homebrew.mitre.org>
Subject: BJCP get's an F in judge training.

Jim Ellingson writes:
>This is a bit long, and it's a bit of a rant. 

Indeed it is.  Thanks for the warning. <g>  Maybe the same should apply to 
this reply--you decide.

>The scores seemed rather low, (to be far more polite than I'm
>feeling), so I did some number crunching on the percentages.
>If I use a straight percentage of 90-100% being an A, 80-90 a B
>and so forth, I get:
>
>        Grades / Scores
>   A's  B's   C's   D's    F's     GPA     Total  Grade Points
>   90's 80's  70's  60's   <60

Except that you have exactly *zero* reason to compare scores on the BJCP 
Exam with academic grades.  As far as I can see, there is little in common 
with an exam of a professional type expected to show mastery and a test 
given after a course.

For instance, let's take the SAT that is supposed to show your academic 
level of mastery.  If you needed to get above 1500 to get an A, how many A's
do they give out?  Are they ripping people off?  I see a lot more similarity
between the BJCP exam and SAT or GMAT or LSAT or bar exam or whatever.

>"Talk on the street" is that the BJCP grading is tough and getting
>tougher.  While the reports from Dennis and Scott are that the
>average scores are quite stable, these average scores are also
>very low.

But what scores do you *want* people to get?  Do you want 20% of people
who take the exam for the first time to demonstrate Master status (getting 
above 90)?  Why?

While I should expect a teacher to teach well enough to have 20% A's as a 
general rule, that does not mean that 20% of exam takers should demonstrate 
Master status.

>Sometimes the hassels of judging/competitions/brew clubs outway the
>pleasures.  I look at my BJCP prep class and see some of the best brewers
>in the club and in the region.  These people do well in the Minnesota
>Brewfest, a large regional competition with 300+ entries, best of show
>judged by Micheal Jackson.  They have very high standards for their hobby,
>and are putting in the time and the money to become BJCP judges.  In return
>for their efforts, they will, on average, get a 65 on the exam.  This reeks
>of Ivory Tower elitism.  I see no justification for the current grading
>scheme.

Well, good brewers may not be good judges--different skills.  And hopefully 
those who have a good BJCP prep class will do better than average--remember 
that lots of those who take the BJCP exam have never had a prep class.  In 
fact, this is a relatively new idea.  When Rick Garvin taught me in a prep 
class several years ago, I could find no one else on the net who had had 
such a wonderful opportunity.

>Our main goal as Judges is to provide feedback to the brewers.

And to further the art of brewing at home.

>The BJCP program exists to train and evaluate these judges.  Since, on
>average, we're turning out C/D level judges, we're failing misserably in
>our training mission.

This is clearly wrong and misleading.  We are not training any judges 
(though it would be nice if the BJCP could start such a process, possibly 
working on a standard course outline for volunteers to teach in their area, 
etc.).  We are simply providing a difficult exam to ensure that those we 
give official BJCP judge status to have shown a broad and detailed knowledge
of brewing and judging.

Thus, anyone who passes (60 or better) should be given a slap on the back 
and added respect.

I think that any system that made it very easy to pass would be a disservice
to the judging and brewing communities.  What would be the point?

Better to work on training to produce more people who can get 80 or above on
the current exam than to make the current exam easier to get an 90 on.


John DeCarlo, MITRE Corporation, McLean, VA--My views are my own
Fidonet:  1:109/131  Internet:  jdecarlo at mitre.org


------------------------------

Date: Wed, 24 Jan 1996 09:16:15 -0500
From: Spencer W Thomas <spencer at engin.umich.edu>
Subject: Re: Historical Braggot with Hops 

One minor error in my post.  I stated that the existence of Digby's
recipe for braggot with hops was proof of the use of hops in 15c
brewing.  Of course, the 1600s were the 17th century, not the 15th.

Of course, this recipe still shows that braggot brewing did not die
out after the 15c, and that there are published historical examples
using hops.

=Spencer Thomas in Ann Arbor, MI        (spencer at umich.edu)

------------------------------

Date: Wed, 24 Jan 1996 09:27:39 -0500
From: Spencer W Thomas <spencer at engin.umich.edu>
Subject: Re: BJCP get's an F in judge training. 

I would assert that the BJCP scoring curve should be closer to that of
a test like the SAT than to the accepted norm in undergraduate
education.  We're trying to separate the "sheep" from the "goats".
That is, those few who score above 90 (an "A" in your terminology,
more like an "A+", in my opinion) should show exceptionally complete
knowledge, not just significant mastery of the material.

In my experience with helping train two sets of judges to take the
exam (a total of about 20 people), the people who failed the exam were
NOT yet qualified to be a judge.  It's really not hard to get a 60 on
the BJCP exam, if you've done a modicum of studying.

I don't believe that either Dennis or Scott proposed that they
*adjust* the scores to fall onto the curve.  They are simply noting
the historical curve.

An anecdote in support of this assertion is that, the year I took the
exam, our proctor told us, after the scores were back, that Pat Baker
had commented that our group had one of the best sets of scores he
had ever seen.  We had held monthly study sessions for 6 months
preceding the exam, and it clearly showed in the results.  Only one
person out of the group of 12 failed, and two of us scored in the
80s.  The person who failed admitted that he had not been ready (he
had started in the study group late, attending only two of the
meetings).

=Spencer Thomas in Ann Arbor, MI        (spencer at umich.edu)

------------------------------

Date: Wed, 24 Jan 1996 10:04:31 -0500 (EST)
From: Fred Hardy <fcmbh at access.digex.net>
Subject: Re: Historical Braggot with Hops


Spencer Thomas correctly points out that by the late 17th century hops 
had appeared in English recipes for beer, mead and braggot. Digby's 
recipes reflect the practice, confirmed by other historical sources, of 
using very small amounts of hops. We also know from Digby that at least 
one person made braggot in the late 17th century. My contention that 
commercial braggot disappered after the Middle Ages is no less valid.

Commercial Porter also disappeared in England, but I suspect someone 
still made it at home. Braggot would have disappeared for the same 
reason: it was no longer commercially viable. Pale malt and the 
saccharometer helped to kill porter, and, IMO, the preservative power of 
hops and a scarcity of honey helped to kill braggot.

Few actual recipes survived the Middle Ages. In 1477 the first book was 
printed in England, and by the 16th century cookbooks with recipes 
claimed to be Medieval came into existence. The authors used ingredients 
that they and their readers could recognize. These were, therefore, 
ingredients which were common at the time the book was written. The use 
of sugar is a prime example. It appears in Medieval recipes 
published in the 17th century, but sugar was very rare in Medieval 
England, and used primarily for medicinal purposes.

Hops were brought to the British Isles as food by the Romans late in the 
1st century before Christ. They could have been in 1st century 
braggot, even though that would be contrary to general practices at the 
time. I will grant that hops are permissible in braggot if we also agree 
that the AHA zero IBU guideline is reasonable - i.e., detectable hop 
levels are not to style, and, as a flaw, ranks alongside high diacetyl 
and esters in Pilsners.

Cheers, Fred

==============================================================================
We must invent the future, else it will  |          <Fred Hardy>
happen to us and we will not like it.    |
[Stafford Beer, "Platform for Change"]   |    email: fcmbh at access.digex.net


------------------------------

Date: Wed, 24 Jan 1996 10:32:54 -0500
From: gwk at world.std.com (Greg Kushmerek)
Subject: Re: JudgeNet Digest #1203 (Jan 24, 1996)

Fred sez:

> There are many reasons why this request is unreasonable, not the least
> of which is your unique definition of vegetarian

That there are different kinds of vegetarians is a good point.

> There are also judges who are strictly kosher, some are vegetarians
> who permit eating fish, others must stick to a salt free diet, and on
> and on

This too is a good point. However, your conclusion is a bit
far-reaching.  Food at a public event is always a compromise.  Some
people don't like spicy stuff.  Some don't like mushy veggies; some
don't like claggy sauce dishes.  Just because you can't fit everyone's
needs doesn't mean you can't try.  In trying you'd probably get the
appreciation of a few others who benefit.

I'm not saying organizers should cater to individual needs; they don't
now.  I've found that you can please the widest number of tastes and a
bit of dietary interests when you keep a few base items.  How tough is
it to make a beef dish that has the rice on the side?  A lasagne with
no meat?  I have friend who runs a restaurant that most of the week
has one vegetarian dish that meets the majority of his vegetarian
customers.  He doesn't go wild reaching that goal either.

OK, the lasagne may have cheese and the lactose intolerant person
can't eat it, but they couldn't have in the first place if it had
beef.  The point being that you can always refuse an idea because you
bog down in the minutae of why it won't work.  

But this gets off the original plea: I asked that people consider a
vegetarian dish.  That's all I asked.  I didn't demand it, and a 'no
way' attitude is just that: an attitude.  If someone feels it is
possible and they do it, then they've done it.  If they can't or don't
want to, then that's that.  By asking, though, I've raised the issue
as a point of awareness.

Please don't dismiss the idea out of hand as unreasonable from
some sense of impossibility.

Thanks,

- --gk

PS - please don't bother following up to this on the list.  I'll take
any replies personally.

- -- 
Greg Kushmerek
gwk at world.std.com

(alter-ego in trunk)
 

------------------------------

Date: Wed, 24 Jan 1996 09:50:30 -0600 (CST)
From: russell at spdc.ti.com (Steve Russell)
Subject: Comments on the grading rant

Hi gang,

Jim Ellingson commented on the score distribution for the BJCP exams:

> The scores seemed rather low, (to be far more polite than I'm
> feeling), so I did some number crunching on the percentages.
> If I use a straight percentage of 90-100% being an A, 80-90 a B
> and so forth, I get:

I'll have to take your word on how the U of MN MechE dept grades.
After grading in one capacity or another (mere grader, TA, instructor,
respectively) at 3 different engineering departments (MIT, Cornell,
Arizona State), my experience is straight scoring as you describe isn't
common.  At least, it's only 1 option, and may or may not be relevant.

> So, by any of the four methods, it seems that most of the
> people are falling into the C/D range.

Regardless of your so-called method, your definition of the C/D range
is what is creating this "analysis".

When I was in school pre-college, these were the grade definitions:
A=excellent, B=good, C=average, D=poor, F=fail.  Is this any better?
No.  But _you_ have to justify why A=Master, B=National, and so on.

> Perhaps a brief discussion of grading is in order.  As Ed Wolfe
> pointed out to me, you can take a norm-referenced perspective
> (i.e., comparisons of people within a population) or a
> criterion-referenced perspective (i.e., comparisons of people
> to standards of mastery).  I suspect lots of heat from
> the latter camp, but, as you'll see, I'm sending it right
> back at 'em.  Also, as a part time educator in a fairly

I am not sure to which camp I belong.  I like to camp, does that count?
Having been a full-time educator, I do feel the need to comment.

> objective discipline (mechanical engineering) I find
> "standards of mastery" to be subjective and a bit of a cop-out
> by the instructor.  (e.g. "The instructor is perfect, gave the
> perfect lectures and tests.  Students fell short so tough.")

I believe that engineers (myself included) are particularly susceptible
to this illusion of objectivity.  "Two plus two equals four so
there you're wrong if you say five."  In the real world, judgements
must be made.  If the BJCP believes that only 2/3 of the bums taking
the test ought to pass, that seems well within the societal norms.

In any case, who is to say that if students score below 90, and I
feel that 90 is passing knowledge, that I can't fail them?  (Other than
my department head and dean, of course).  Or if I deliberately make
difficult tests, couldn't a 33 be an 'A'?  Sound ridiculous?  I have
taken tests where the average scores were 91, 92 and 25 (undergrad).

> Here at the U of M, the undergrad GPA in the Institute of Technology
> is 2.7 and in Mechanical Engineering it's 2.9.  In the Graduate

At Cornell and ASU, it was lower.  At MIT, higher.  Any reason why
we should prefer your university over all others?

> School, students must earn at least a B on courses in their
> own department, at least a C in courses not in their department
> and must maintain at least a 3.0 GPA.  I suspect these numbers
> and rules are typical.

Yes, but the theory is that we expect "Master" level performance
of our graduate students.

> Scott goes on to mention that the scores rise by 5 to 10 points
> for retakes.  Well, if you're taking the exam a second time,
> you're looking at a 2.0-3.05 GPA.  This is also very low.

Again, _you_ are the one assuming criteria for reference.
I could define 70 as an 'A' (I have in my courses before) and
the GPA would be closer to 4.0.  Neither way has any meaning.

> "Talk on the street" is that the BJCP grading is tough and getting
> tougher.  While the reports from Dennis and Scott are that the
> average scores are quite stable, these average scores are also
> very low.  I know of a recent examinee who is a professor in
  ^^^^^^^^
See above.  You've defined low and high here, not them.

> a biology field, and who has authored an article for Zymurgy,
> who scored in the 80's.  I'm wondering where the graders earned
> their PhD's from....  Of course they don't offer PhD's in
> penmanship or anal retention....

Well, I know Scott got his from Cornell.  I'm biased on that part.

This is one of the cheapest shots I've seen on this Digest.  Shame.
Go sit in the corner with no dessert until you behave like an adult.

In any case, anal-retention has a hyphen.

> I propose that the BJCP modify it's grading structure
> to conform with accepted university practices.

Sure.  Let's poll everyone who has contact with various universities,
instead of just adopting the U of MN's practices, however.  Include
all the weed-out courses as well as those graduate courses you mention.

> statistics don't support this position.  A test or course in which 
> the average grade is somewhere between a D/C and a C is one tough 
> (aka narrow, rigid, inflexible, anal retentive) test.

You may very well be correct but again this depends on your definition
of what constitutes A,B,C,D,F.  You have to justify that before you
can waltz in here spewing forth this kind of dribble.  Wipe your
mouth and go back to grading dynamics quizzes.

Regards,
Steve

Stephen W. Russell				995-3928		office
Multilevel Interconnect Technology		995-2770		fax
Texas Instruments				597-6967		pager
Semiconductor Process and Device Center		340-2574		home


------------------------------

Date: Wed, 24 Jan 1996 10:57:21 -0500
From: bosko at cyberportal.net
Subject: Food for Thought

Fred Hardy responds to Greg Kushmerek:

At the risk of seeming callous, Greg, no way!

There are many reasons why this request is unreasonable, not the least of 
which is your unique definition of vegetarian. There are also judges who 
are strictly kosher, some are vegetarians who permit eating fish, others 
must stick to a salt free diet, and on and on.

We would have to put dietary requests on the judge registration form to 
satisfy all food fads and health driven needs (beliefs?). Please, 
Greg, we organizers are running a homebrew competition, not catering a 
wedding.
__________________________________

Pretty callous, Fred.  Greg's point is a good one.  He's not asking for a 7 
course meal, he's only asking for a plate without meat.  Spaghetti would do.  
This not complicated or inconvenient.  Here in the east there are a lot of 
judges who would rather not eat meat, either as a general rule, or as a judging 
rule (fat on your mouth or whiskers kills the head of the beer).  Maybe in 
Podunk they think its a fad, but in some places people see it as a life 
commitment.

Nastrovia
Julian Zelazny

"I'm old and I'm falling apart!"  -  Former Supreme Court Justice Thurgood 
Marshall, when asked why he was retiring from the Court.

------------------------------

Date: Wed, 24 Jan 1996 11:06:38 -0500 (EST)
From: Scott Bickham <bickham at dave.nrl.navy.mil>
Subject: Grading ranks

Sorry to have not participated in the discussions that my posting
prompted, but the last JudgeNet I received was last Tuesday.  I
have given Chuck my new e-mail address and have also resubscribed,
but until then I would appreciate being forwarded copies of
JudgeNet.  Are the digests being archived anywhere?

Thanks,
	Scott
- -- 
====================================================================
E-Mail: bickham at dave.nrl.navy.mil		FAX:(202) 404-7546
						
Office: Naval Research Laboratories		Home and BJCP:
	Code 6691, Complex Systems		7507 Swan Point Way
	Washington, D.C  20375			Columbia, MD  21045
	(202) 404-8632				(410) 290-7721
====================================================================

------------------------------

Date: Wed, 24 Jan 1996 12:08:07 -0500 (EST)
From: Scott Bickham <bickham at dave.nrl.navy.mil>
Subject: Comparisons of exam scores and letter grades

Steve copied me on his response to Jim Ellingson's posting, so
hopefully by responding I can catch up on the conversation.  See
my above posting to see why I have been silent.

To make a comparison with judging ranks and letter grades in an
institution, you should start with the same definitions.  It's
clear that one needs a 60 to pass the BJCP exam, while C is
generally the passing grade for a course.  Scores in the 70s
would then be similar to B's and scores in the 80's are A's.
BJCP exam scores in the 90's are, and should be tough, to 
achieve, and the best analogy would be an A+.  These people
demonstrate exceptional knowledge of beer styles, brewing and
have well-developed tasting skills.  Of the approximately 20
Master judges, there are some names you should recognize: Scott
Birdwell, Mike Fertsch, Dan Hall, Jay Hersh, Al Korzonas, Steve
Stroud, Chuck Cox, Darryl Richman, Gregg Smith and Betty-Ann
Sather.

I scored an 86 on the first attempt at the BJCP exam, and scored
above 90 on a retake last January.  I have built up a decent
library of brewing and beer books and logged in over 60 hours
re-reading eveything and taking many pages of notes.  On beer
styles, I relied on Michael Jackson's books and the Classic Beer
Styles series, while my technical background was refreshed by
reading Miller, Noonan's "Brewing Lager Beer", Fix's book on
brewing science, both volumes of "Malting on Brewing Science"
and "The Practical Brewer".  The reading was supplemented by
judging experience in 25 homebrew competitions and a tasting log
that lists nearly 1000 commercial beers.

Jim Ellingson wrote:

> > Scott goes on to mention that the scores rise by 5 to 10 points
> > for retakes.  Well, if you're taking the exam a second time,
> > you're looking at a 2.0-3.05 GPA.  This is also very low.

This is bad paraphrasing.  I think I said that that exams generally,
i.e. on the average, increase by 5-10 points.  The learning curve
is probably logarithmic, as it is for many subjects.

Jim adds:

> > "Talk on the street" is that the BJCP grading is tough and getting
> > tougher. 

Those are dangerous streets to be on.  The average BJCP exam score
is in the low to mid 70s, with a failure rate of less than 5%.  If
anything, the grading has become more lenient, but we are not doing
it at the expense of lowering the standards and inflating the grades.

Jim gives an example:

> > average scores are quite stable, these average scores are also
> > very low.  I know of a recent examinee who is a professor in
> > a biology field, and who has authored an article for Zymurgy,
> > who scored in the 80's.  I'm wondering where the graders earned
> > their PhD's from....  Of course they don't offer PhD's in
> > penmanship or anal retention....

I agree with Steve that this is a cheap shot.  Having a PhD does not
make one qualified to be a Master Beer judge.  Half the the written
exam is on  beer styles, while 30% of the exam consists of being
able to accurately judge beers.  Last I heard, these areas were not
covered in Biology courses, but correct me if I'm wrong.  Similarly,
my Ph.D in Physics was of little help but as you can read above,
I supplemented my education with lots of beer-related activities.

Jim also managed to get in a shot at the BJCP newsletter, which was
put together, printed and mailed almost single-handedly by Dennis
Davison.  Dennis will be the first to admit that his grammer and
spelling are less than perfect, but I for one am grateful for the
newsletter.

> > I propose that the BJCP modify it's grading structure
> > to conform with accepted university practices.

I really think you need to look at what the scoring and knowledge
relationship is.  My analogy is this:  suppose that that you
have a group of people whose knowledge ranges from Associate degrees
to Ph.Ds, with Bachelors and Masters in between.  The goal is
to distinguish between them with a single exam, and that is the
purpose of the BJCP exam.  Based on the ability demonstrated in
the exam, we need to determine the Recognized, Certified, National
and Master levels.  Your university analogy is based on an exam
given to a single group of students, with the level of the exam
geared to that particular group.  You would be better off imagining 
an upper level graduate exam given to a group of Ph.D., Master, 
Undergraduate and maybe even High School students.  Do you still feel 
like the standard grading scale would apply?  I didn't think so.

Yours,
	Scott Bickham
	Exam Director, BJCP
- -- 
====================================================================
E-Mail: bickham at dave.nrl.navy.mil		FAX:(202) 404-7546
						
Office: Naval Research Laboratories		Home and BJCP:
	Code 6691, Complex Systems		7507 Swan Point Way
	Washington, D.C  20375			Columbia, MD  21045
	(202) 404-8632				(410) 290-7721
====================================================================

------------------------------

Date: Wed, 24 Jan 1996 13:11:32 -0500
From: drbeer at tiac.net (drbeer)
Subject: re:BJCP get's an F in judge training.


A recent post gives a long statistical evaluation of scores.  However I
believe it makes a mistake in comparing BJCP examinees to college
populations.  First off I think that the nature of the testing is different.
I don't believe the BJCP generally intends to grade along a curve.  In
college testing the assumption is that the relative distribution of
capabilities of the students is fairly similar from year to year and that
the tests are not necessarily a reliable indicator of knowledge.  This leads
to grading along a curve with an inherent distribution of individuals into
different portions of the curve.

With regard to the BJCP exam this is different.  While most BJCP test takers
study hard and are quite conscientious, the nature of the material is in
itself inherently different from college exams.  In a college exam all
material covered on the exam is present in the course and while difficult,
the assumption is that it is a matter of simply mastering the material and
demonstrating this skill on the exam.  By its very nature, i.e. the
familiarity with a world's worth of beer styles, the material covered on the
BJCP exam is very difficult for all but the most studied and well traveled
of examinees to have covered.  Thus by their very nature the test takers
begin with somewhat of a handicap.  The test is designed to be comprehensive
of the material.  By that I mean it is intended to ascertain the skills of
both very experienced individuals as well as inexperienced novices.  Since a
very small proportion of test takers will be very experienced then it
certainly makes sense that only a very small proportion score well i.e.
above a 90.

Think of the test as a gating factor.  It determines whether a person has
enough knowledge to begin in the program, and how far  that person should be
able to progress.  As such the test is more of a proof of basic competency.
I recognize that even those who score high and have participated in the
program and accumulated many experience points still always have something
to learn. 


I find that the criticism of the test from many individuals commenting here
seems to me to be mostly unfounded.  How many of those criticizing the test
have taken it more than once?  How many of them have graded tests, and
actually read examinees answers and judged them against the questions asked?
I think many comments are based on a rather minimal experience with the
test. On one hand many complain that the test is too hard, yet I just as
often hear complaints about poor judging at competitions. So what is the
problem? Is the test too difficult? If so then why are there complaints
about judge quality?? Is it too easy?? Then why do others complain that it
is too hard?

I believe the test does a very good job.  It is a difficult test, as well it
should be.  When you consider that most test takers are only knowledgeable
in a fraction of the material (whose range is quite large when you consider
the entire world of beer) you find that the scores given reflect this fact
in that they do tend to cluster towards the low end. That simply reflects
the reality of the skills of the test takers compared to the enormous amount
of material out there.  When individuals enter the program they generally
first begin to expand their experiences more aggressively by judging
competitions, attending beer tastings and traveling more widely.  While  I
have no statistical proof on this point my anecdotal experience of many
years is that this is the case.

While I don't contest that there may be room for improvement in the
questions and even structure of the test I believe that overall the test is
fundamentally sound.  While the possible consideration of a radical change
in test structure to separate the test into two tests might be merited there
are also significant drawbacks to such an approach.  For example an entrance
exam which would allow individuals to enter the program and perhaps advanced
as far as certified level, and an advanced exam which would be significantly
more difficult and would be used to determine those who could progress on to
national and master,  is one possible structure.  However this would have
the drawback of ensuring that everyone would have to retake the exam.  As it
stands today a great majority do but not everyone.

Personally the only changes I would advocate on the test are consistent with
my understanding of some of what is currently being proposed, i.e. addition
of some multiple choice questions to replace some of the short answer essay
questions, and moving the tasting portion to a single block of time at the
and of the test.
With regard to level of difficulty I think it's just right.

I personally would advocate that many of those criticizing the current test
defer their criticism long enough to have an opportunity to grade some
tests.  I think that they would gain a different perspective on how
difficult the material is and what the skill level of the test takers
themselves are after doing some grading.


                          Jay

- ---------------------------------------------------------------------------
Hopfen und Malz, Gott erhalt's

This is a key free document, no keyboards were harmed in its creation.

If Bill Gates is such a visionary how come he couldn't envision that
computers would have over 1MB over memory when he began selling MS-DOS over
a decade ago??


------------------------------

Date: Wed, 24 Jan 96 11:20:48 CST
From: korz at pubs.ih.att.com (Algis R Korzonas)
Subject: Alts/Judge Training

Jim writes:
>What you are doing is stating a preference for highly bitter alts as
>exemplified by those made in the hausbraueries of Dusseldorf.  This
>does not mean that a 30 IBU alt is not a true alt.  The range of 
>IBUs for alts clearly is large, including 30-55 IBUs.  Its hard
>to argue that a 35 IBU alt is lacking in qualtiy over a 55 IBU one.
>Its a matter of brewers choice and the fact that the vast majority
>of alts in Germany are in the 30-45 IBU range should not be ignored.

I guess I did show my preference for the highly-bitter Alts, but I'd
like to share the blame with Roger and Michael, as I'm not the only
one that feels that the hausbraurei (Brewpub) Alts are the best.  As
you well know, the more bitter Alts are brewed by some stubborn, very
traditional brewers and who are far more interested in brewing what
they feel is best than catering to a wider audience.  The larger breweries
(especially the ones that must rely on selling beer outside of Duesseldorf
to survive) have to try to appeal to a more beer drinkers.  Therefore,
they have to be less agressively hopped.  If we make the same mistake
as the AHA and go "by the numbers" as opposed to noting the traditions,
then we continue to dilute the style.

My main points, which I guess I failed to highlight in my previous email, is
that while we cannot ignore the fact that there are many Duesseldorfer Alts
that are hopped in the 30-45 IBU range, we *must* not mistake Muenster and
Dortmunder Alts (most of which are in the 20-35 IBU range) for Duesseldorfer
Alts and that the AHA (and BJCP!) guideline should instead contain the 
statement:

  "Duesselforf-style Altbiers should not be confused with Muenster or
   Dortmund-style versions which are entirely different beers, the
   latter not even an ale.  While the majority of commercial *Duesseldorfer*
   Altbiers are brewed with 30-40 IBUs, the most traditional ones have 45-60
   IBUs."

>Similarly, the vast majority of Bavarian Pils are 30-35 IBUs, but we
>may actually prefer more bitter ones.  This does not mean a Pils must
>be 40 IBUs to be worthy of a high score.

I'm not saying it does.  On the other hand, your range is far smaller than
the one in the AHA guidelines for Duesselforfer Alts.  There is virtually
*no* similarity between a 25 IBU Alt and Zum Uerige at 55+.  Try entering
Miller Genuine Draft into the Bohemian Pilsner category.

***
Jim writes:
>Subject: BJCP get's an F in judge training.
>
>"Talk on the street" is that the BJCP grading is tough and getting
>tougher.

I don't know about that.  First time (in 1992) I got a 78, second time
(in 1994) a 90.

>Our main goal as Judges is to provide feedback to the brewers.
>The BJCP program exists to train and evaluate these
>judges.  Since, on average, we're turning out C/D level judges,
>we're failing misserably in our training mission.

As far as I know, the BJCP has never pretended to be a training organization.
Perhaps now it will take on that role and I would be happy to contribute
my time if needed, but it is unfair to blame the BJCP for failing at training
judges.

Up till now, the learning has been the responsibility of the examinees.
In 1995, I taught exam preparation classes for several weeks previous to
and then administered the exam.  There is an incredibly wide scope of
knowledge (not just biology, so a PhD in biology can only help you so much)
that is necessary to do well on the exam, so I cannot blame examinees for
missing some questions.  On the other hand, in my prep class I know that
I gave suitable answers for several questions that later (by chance) happened
to be on the exam -- skimming the answers of some of my students, I found
that several of them still got the answer backwards or completely off-base.
Am I a lousy teacher?  I don't think so.  Was there just too much material
to cover?  No, because some people can obviously master it.

I don't think that the BJCP exam should be compared to a college exam.
Perhaps it would be more fair to compare it to some other graduated-rank
certification exam?


Al.

Al Korzonas, Palos Hills, IL
korz at pubs.att.com

------------------------------

Date: Wed, 24 Jan 1996 15:37:57 -0500 (EST)
From: Craig Pepin <ckp at acpub.duke.edu>
Subject: testing, and more

Jim Ellingson's long (self-described) rant can be demolished by one very simple
point:  What is the connection between the BJCP test scores and the ABCDF
grading system?  Answer - absolutely none, and there is no reason why there
should be, either.  Once this obvious fact is pointed out, none of his
criticisms hold water.  But if I may be allowed to briefly flog some of the
subpoints contained in this large dead horse ...

>I know of a recent examinee who is a professor in
>a biology field, and who has authored an article for Zymurgy,
>who scored in the 80's.  I'm wondering where the graders earned
>their PhD's from....  

I'm currently earning mine from Duke, but that is beside the point.  What is
important here is that a) 80's is a damn impressive grade for a first time
taker, (placing him/her in the top 20% of all exam takers) and b) a Ph.D. may or
may not have a great influence on whether a person is a good judge, at least
until Davis starts granting Ph.D.s in beer evaluation.  Two different contexts.

>I propose that the BJCP modify it's grading structure
>to conform with accepted university practices.

Why?  What's the point?  Simply accept that the BJCP operates on a different
scale than your average grade-inflating university and be done with it.  The
problem here is not the grading format, but your perceptions of how it should
be, which you draw from a different field of human endeavour.  Again, two
different contexts.  Adjust your perceptions, not the system!

My  .02 cents on two other topics:

I ranted about big beers last year when two obscenely out of category  beers -
Bitter and Scottish Export - won at 94 nationals.  However, I don't think this
is an entrant problem as much as judge problem:  I would hope that a judge 
would be able to tell if a beer has twice as much alcohol as it should, even 
if theother flavor constituents are in balance for style - the alcohol and or
maltiness takes it out of style.  I'm not sure I agree with John DeCarlo
(although we've met, I've never judged a flight with him, so I guess I'm not 
one of his datapoints) either:  I don't expect a judge to recognize a 1.066 
Fest as being out of style (though the best judges I have judged with can) but
I do expect them to accordingly take off points for a special bitter with 7 
1/2% alcohol.  Such a beer may still win the category if there is nothing else
better, but it should not score higher than a 30.  And the same thing goes for
120 IBU IPA.  It is out of style, not because of the technical info, but 
because there is no way a beer with such parameters could taste like the style 
it is supposed to emulate.  

As far as Alts go, they have already been well covered by some knowledgeable
writers, so I will merely add that when I went to Dusseldorf this summer, I
found Zum Uerige to be substantially hoppier than any other Alt available.  It
was also the most popular place, which may help account for why it is usually
held up as *the* preeminent example of the style.  Personally, I preferred Im
Fuchschen's offering, which is what I try to emulate when I brew an alt.  
Since I now have some experience with this difficult style, I always 
volunteer to judge it when I go to competitions (unless I'm entering, of 
course).  But it is worth noting that very few alts besides Zum Uerige, 
*including* the Dusseldorf Alts, consistently get into the 40 IBU range 
or higher.

Craig Pepin
Triangle Unabashed Homebrewers
Durham, NC


------------------------------

Date: Wed, 24 Jan 1996 13:36:54 -0800 (PST)
From: Jim Cave <CAVE at PSC.ORG>
Subject: Jim Ellingson's comments on judge training.....

	I'm not usually the sort of person who normally complains about
postings on the Judgenet, but here goes.....

	Was anyone else, other than me, thoroughly confused about Jim 
Ellingson's comments on judge training?  All I got out of it was stuff on
grading and GPA's.  I seemed to have missed the point to his post.  All I 
can guess is that Jim was dissatisfied with his mark of 65%.

	Jim, perhaps your background in Mech Eng has not left you sufficiently
prepared for essay writing...or perhaps with an understanding of the biology of
beer.  I  recall no suprises when writing the exam.  I expected essay 
questions and I got them.  Budgeting your time is obviously critical when
completing this type of exam.  Also important is staying on topic--and Jim, 
you seemed to have trouble with that, judging from the rambling nature of
your post.

			Jim Cave

------------------------------

Date: Wed, 24 Jan 1996 15:27:49 -0800 (PST)
From: Mark Taratoot <taratoot at PEAK.ORG>
Subject: judges with dietary restrictions

First, let me preface this article with an apology to the JudgeNet Digest 
community.  This is not about judging, and it is only tangentially about 
competition organization.  However, this is something I would like to 
address.  Please excuse the bandwith.

I am responding to an article by Fred Hardy in the January 24 JudgeNet 
Digest.  Fred was responding to a request by Greg Kushmerek that 
competition organizers consider an alternate lunch for judges (and 
participants?) that would be suitable for people on vegetarian diets.  

Fred writes:

> At the risk of seeming callous, Greg, no way!
> 
> There are many reasons why this request is unreasonable, not the least of 
> which is your unique definition of vegetarian. There are also judges who 
> are strictly kosher, some are vegetarians who permit eating fish, others 
> must stick to a salt free diet, and on and on.
> 
> We would have to put dietary requests on the judge registration form to 
> satisfy all food fads and health driven needs (beliefs?). Please, 
> Greg, we organizers are running a homebrew competition, not catering a 
> wedding.

Actually, the solution Greg suggested would solve all the problems. 
First, a person who observes "strictly kosher" dietary restrictions would
likely not be judging home brewed beer for fear of violating the kosher
dietary law.  I doubt many homebrewers have certified kosher kitchens. 
Strictly kosher judges would only be able to judge "strictly kosher" beer. 

Regarding the vegetarians who ALLOW fish, this does not mean that the eat
ONLY fish.  Salt free is a bit of a problem, but can be accommodated as
well.  As I recall, Greg suggested beans and rice as a substitute food. 
This would pass muster for vegetarians, vegans, lacto-vegetarians,
ovo-vegetarians, fish-eating vegetarians, fowl-eating vegetarians,
hamburger-eating vegetarians, those "strictly kosher" judges who have no
problem drinking beer of dubious origin, and, if salt is left out of the
recipe, salt free diets as well.  Also, this dish would be acceptable for
people using MAO inhibitors, those people who are lactose intolerant
(which is also a problem for judges IF lactose is used in brewing), 
those with wheat allergies, dairy allergies, etc. 

Of course, there will always be an exception, but a simple offering like
beans and rice would satisfy most diets.  It is an inexpensive alternative
that is also healthy and does not cloud the palate or kill a good 
head like a bratwurst can. 

Agreed it does take a bit of extra effort on the part of the 
organizer(s).  However, I imagine that each club has at least one 
vegetarian who would agree to be delegated this responsibility.  This is 
an easy way to improve competitions.  We can not have competitions 
without judges, and anything that we can do (especially something as easy 
as this) to improve the judging experience is effort well invested.

Rant mode off.

Again, please excuse the bandwith.  Now back to the high quality we are 
accustomed to on Judgenet.

- --
Mark Taratoot            "...though my problems are meaningless,
taratoot at peak.org         that don't make them go away."     
                                                         -Neil Young

------------------------------


End of JudgeNet Digest
************************