Return-Path: synchro!judge-owner at uu6.psi.com
Received: from srvr8.engin.umich.edu (root at srvr8.engin.umich.edu [141.212.2.81]) by srvr5.engin.umich.edu (8.7.3/8.7.3) with SMTP id CAA23693 for <spencer at srvr5.engin.umich.edu>; Tue, 30 Jan 1996 02:42:03 -0500 (EST)
Received: from redheat.rs.itd.umich.edu (redheat.rs.itd.umich.edu [141.211.83.36]) by srvr8.engin.umich.edu (8.6.12/8.6.4) with ESMTP id CAA11359 for <spencer at engin.umich.edu>; Tue, 30 Jan 1996 02:39:21 -0500
Received: by redheat.rs.itd.umich.edu (8.6.12/2.2)
	with X.500 id CAA06980; Tue, 30 Jan 1996 02:39:20 -0500
Received: from uu6.psi.com by redheat.rs.itd.umich.edu (8.6.12/2.2)
	with SMTP id CAA06974; Tue, 30 Jan 1996 02:39:17 -0500
Received: from synchro.UUCP by uu6.psi.com (5.65b/4.0.071791-PSI/PSINet) via UUCP;
        id AA10540 for ; Tue, 30 Jan 96 02:31:57 -0500
Received: by synchro.com (smail2.5)
	id AA17339; 30 Jan 96 01:18:34 EST (Tue)
To: judge-recipients at synchro.com (JudgeNet Recipients)
From: judge-owner at synchro.com (JudgeNet Administrator)
Reply-To: judge at synchro.com (JudgeNet)
Errors-To: judge-error at synchro.com
Precedence: bulk
Subject: JudgeNet Digest #1207 (Jan 30, 1996)
Date: 30 Jan 96 01:18:34 EST (Tue)
Message-Id: <9601300118.AA17339 at synchro.com>


JudgeNet Digest #1207				   Tue 30 Jan 1996


                               JudgeNet
			 The Beer Judge Digest

                 digest submissions: judge at synchro.com
          administrative requests: judge-request at synchro.com
    send cancellations & rank updates to the administrative address
	  messages sent to the wrong address will be ignored

        WWW Archives: http://www.umich.edu/~spencer/beer/judge

                 Editor: Chuck Cox <chuck at synchro.com>
             Archivist: Spencer Thomas <spencer at umich.edu>

      Publishers: SynchroSystems and the Riverside Garage & Brewery

            Anti-Prohibitionists may also be interested in 
                LiBeerty: The Libertarian Beer Digest
           Subscription info: libeerty-request at synchro.com

     For BJCP General Information contact: geninfo at bjcp.synchro.com


Contents:
  Re: Braggot                              (Fred Hardy)
  Response to Norman  Dickinson            (WILLIAM GIFFIN)
  Re: Braggot                              (Dan McConnell)
  EXAM DISTRIBUTIONS                       (Wolfe)
  More on Testing                          (Wolfe)
  re: rewarding fat beers                  (919) 405-3632" <danz at edasich.rtp.semi.harris.com>
  Rob and Norms Comments                   (Dennis Davison)
  Here is the BJCP exam                    (Scott Bickham)
  Re: rationale for a distribution         (Robert L. Lamothe)

----------------------------------------------------------------------

Date: Mon, 29 Jan 1996 08:20:51 -0500 (EST)
From: Fred Hardy <fcmbh at access.digex.net>
Subject: Re: Braggot 


Spencer Thomas asked if braggot was made in places other then the British 
Isles and possibly Scandinavia. The answer is a definite maybe. This is 
based on documentation from the 2nd century BC which described a Celtic 
strong drink made from grain and honey. These particular Celts were from 
Gaul, and, by implication, had knowledge and practices similar to Celts 
throughout Europe.

Hops were indeed known and used in brewing on the European continent well 
before the 11th century. Whether the brewing practices of the ancient 
Celts (i.e., braggot) survived the early Middle Ages in Germany and the low 
countries is less clear. At the time of the Norman invasion the drink of 
the invaders was wine, but beer was the drink of countries above the vine 
line.

The practice of adding honey to malt beverages seems to have had the same 
goal as adding it to sweet cider. The higher alcoholic content obtained 
by adding the honey gave the fermented cider and beer a greater shelf 
life in the absence of hops. I realize that is not a compelling argument 
that there were no hops in braggot or cyser. There certainly could have 
been even though it seems counterintuitive. There would have been no 
economic, medical or taste (sweet was good) reason for doing so.

Clearly "modern" braggot as entered in homebrew competitions does contain 
hops. I suspect most beer drinkers (judges?) are somewhat put off by 
unhopped alcoholic beverages made with malt. I entered a Medieval beer 
(very strong, but unhopped) as a specialty once, and judges' comments 
said "why did you make this unhopped? It is unbalanced." Scores were 
correspondingly low.

The AHA/AMA guidelines imply that braggot is unhopped. If judges expect 
hops, the guidelines should be changed to provide guidance to those 
brewers who rely on them to formulate beverages they subsequently enter 
into competitions as barggot.

I find the 50% rule of the Mazer Cup competition to be rather amusing. It 
seems arbitrary and capricious. Do the organizers check braggot entries 
and disqualify them if they don't meet the rule? Is it 50% by weight of 
raw materials or fermentables (excluding dextrines)? Pardon me, but I 
really don't know. It seems simpler to me to exclude hopped braggot, thus 
insuring that entries will be mead (if braggot is mead) and not beer with 
an unacceptable honey content. If judges subsequently use the AHA's IBU 
guideline of zero, any beers should be excluded from winning at the 
judges' table rather than by the entry police.

Just a thought from someone who has never entered the Mazer Cup.

Cheers, Fred

==============================================================================
We must invent the future, else it will  |          <Fred Hardy>
happen to us and we will not like it.    |
[Stafford Beer, "Platform for Change"]   |    email: fcmbh at access.digex.net


------------------------------

Date: Mon, 29 Jan 1996 10:25:03 -0600
From: WILLIAM GIFFIN <billgiffin at mci.newscorp.com>
Subject: Response to Norman  Dickinson

Top of the morning to ye all,

My. my. my, a bit on the sensitive side aren't we Norman?  I express a
plausible reason for the migrations to the bigger beer, and you go on and on
about your qualifications to tell why I'm wrong.  Seeing that you are a
sociology major perhaps you will do the scientific study to either prove me
wrong with scientific data or maybe perhaps you will find that Blind old
Bill is correct.  I presented my post of the other day to present a possible
reason for the Big Beer winning thing.  I hope that you didn't break you arm
beating your back Norman on telling how you won with beers that were within
the style guidelines.  But how do we know what the bittering rate was, a bit
too bitter perhaps, nor did you indicate what the terminal gravity was touch
high or low to give the complexity you though would win?  Did you include
ingredients which were inappropriate for the style?  Can you read a
hydrometer?  It appears that's most of the folks how entered the Nationals
couldn't.

I make my point any beer that is out of style should not under any
circumstances get a score better than 24. You said "A good beer brewed out
of style and judged as out of style can often easily outscore most beers in
a flight."  Is this part of the elitist conspiracy that only want BIG BEERS
to win?

This is an ASSOCIATION?  Why is it when we express concerns about this
ASSOCIATION that upsets establishment we get bashed?   

This is a forums to express our concerns that with discussion may improve
the BJCP, may improve the feedback to the brewers, and may continue to be a
rewarding hobby.

One last thought to you Norman related to your concern about the CONSPIRACY.
Are you afraid that this conspiracy will do away with Master judge and you
won't make it.  If all you need is points I offer what ever of mine you need
to make you a Master Judge as I have plenty and they are not worth a DAMN thing.

At the present my pint is less then half full and I shall remedy that by
filling it up.  

Now that I have a full PINT, happens to be a lovely Dunkel, Let's get on
with it and improve the BJCP.

By the by Scot didn't blame the brewers.  To that I say I have meet the
brewers and they are us.

May the road rise to meet you.
		Bill  Giffin
Bill		61 Pleasant St. 
		Richmond, ME  04357
		(207)-737-2015
		billg at giffin.iii.net

All you need is a few good friends and plenty to drink because thirst is a
terrible thing!


------------------------------

Date: Mon, 29 Jan 1996 11:18:58 -0500
From: danmcc at umich.edu (Dan McConnell)
Subject: Re: Braggot

At 8:20 AM 1/29/96, Fred Hardy wrote:
>
>I find the 50% rule of the Mazer Cup competition to be rather amusing. It
>seems arbitrary and capricious. Do the organizers check braggot entries
>and disqualify them if they don't meet the rule? Is it 50% by weight of
>raw materials or fermentables (excluding dextrines)? Pardon me, but I
>really don't know. It seems simpler to me to exclude hopped braggot, thus
>insuring that entries will be mead (if braggot is mead) and not beer with
>an unacceptable honey content. If judges subsequently use the AHA's IBU
>guideline of zero, any beers should be excluded from winning at the
>judges' table rather than by the entry police.


Here is the section of the 1995 MCM announcement describing Braggot exactly
as posted.
- ------
*8. BRAGGOT (BRACKET):  Honey and Malted barley (as much or more honey than
   malt).
- ------

Please notice that there is no RULE that there must be 50% honey.

50% is not a rule, it is a guideline.  Big difference.  As one of the
'entry police' I assure you that we have never excluded a mead from the
competition based on the ingredient list.  A honey *beer* entered in the
*braggot* category probably will not do well, but will not be banned from
the competition. You could enter a wine in the pyment category and do as
poorly. I think that would foolish and a waste of time, money, judges and
expectations.

50% is arbitrary, yes, but seemed like a good *starting point* based on our
experience, to assist the meadmaker and increase the mead character of the
entries.  As one of the organizers, I can change the guidelines.  From my
standpoint the wording of 1995 worked and I see no need to change it for
1996.

It seems simpler to me to exclude braggot.  An alternative is to exclude
the rules. Not a bad alternative.


DanMcC.



------------------------------

Date: Mon, 29 Jan 96 09:57 CST
From: Wolfe at act-12-po.act.org
Subject: EXAM DISTRIBUTIONS

I've been out of town and haven't had time to catch up on the exam thread
until today.  A couple of thoughts:

STANDARDS

Jim Ellingson began some conversation about the score distribution on the
exam.  The problem here is that it's unclear whether the exam is suppose to
be criterion referenced or norm referenced.  That is, it isn't really clear
whether we are expecting a score of 80% to mean that the judge has mastered
80% of the content required to be a master judge or whether an 80% means
that the person scored better than 80% of the other examinees.  It seems
like we should be talking about the former (criterion referenced) rather
than the latter (norm referenced).  If we are talking about criterion
referenced testing (where a judge is showing evidence of mastery), THEN WE
SHOULDN'T BE TALKING ABOUT SCORE DISTRIBUTIONS AT ALL.

An analogy--the SAT is a norm-referenced test.  It's purpose is to rank
order a bunch of people who are applying for the same colleges.  That way,
the colleges can choose to take only the best (according to the test) of
who applies.  The scores say NOTHING about what the people know or are
capable of doing.  With norm-referenced tests, the distribution is
everything.  I disagree with Stephen Russel.  I think that this approach is
a bad idea.  As Jim Brennan and John DeCarlo suggest, we should be
developing a training program that helps judges prepare for a
criterion-referenced test.

An example of a criterion-referenced test, on the other hand, would be
whether a person can complete an obstacle course or not.  If a person can't
climb a wall or jump over a pit, the person can't do what we want them to
do.  They haven't mastered the skills that are necessary to complete the
task for which the obstacle course is screening.  With criterion-referenced
tests, the distribution isn't really relevant unless you're competitive and
want to see who can do the tasks the fastest.  Then it becomes a
criterion-referenced test that norms the people who pass it.

Anyway, if the BJCP exam is a criterion-referenced test (which I think it
should be) the only thing that is important is that the cutoff scores are
valid and reliable--that is, that the test is fair.  I'm not bothered by
the fact that there are so few 90s on the exam as long as we can be ensured
that the people who are scoring in the 90s are, by far, the best and most
qualified judges.  Scott Bickham provides some testimony that this is so.
But, it's the responsibility of the exam committee to provide more than
just testimony that this is true.  To my knowledge, information to confirm
this isn't available.

Ed Wolfe
wolfe at act.org

------------------------------

Date: Mon, 29 Jan 96 10:31 CST
From: Wolfe at act-12-po.act.org
Subject: More on Testing

No pun intended for the subject line ;)

Craig Pepin and Sullivan51 at aol.com brought up a few points about the
reliability of the BJCP exam.

Craig says that interrater reliability is probably an area where the BJCP
exam can be improved.  Although the information isn't available (have I
said this enough times yet?), Craig might be right.  I think that the steps
that Scott Bickham described in response to my original post are great and
will probably raise the level of grader agreement.  However, even with the
most highly trained scorers, there is typically one or two in a large pool
of raters who score consistently higher or lower than others in the pool.
This is a problem--but it's a problem that is easily corrected.  It's
pretty simple to run grader statistics to identify who tends to grade more
harshly or leniently.  Once the outliers have been identified, they should
be made aware of their tendencies.  Often, consistent feedback like this
will help graders adjust their scores.  I doubt that this is currently
being done.  If this does not help the situation, it's very easy to adjust
scores assigned by these graders by simply adding or subtracting their
effect out of the scores they assign (I'm making the mathematics sound a
little simpler than they really are here).

In my opinion, though, a bigger problem is probably the fact that we don't
know whether the alternate forms of the test are truly parallel.  I
recently retook the exam.  I think that the test I took the second time had
a more difficult sample of questions than did the first exam I took.  I
don't have any proof that that is so, but then again, the BJCP exam
committee doesn't have any proof that what I am saying is not true.  I've
done hundreds of analyses on open-ended tests.  The overwhelmingly largest
source of "testing error" for these tests is due to item sampling.  That
is, it DOES matter what set of items you get.  The items in a test often
have nearly as much importance as does the ability level of the examinee.
This is a pretty serious problem!  Again, I don't know that this is the
case for the BJCP exam, but I suspect that it is (and there isn't any
evidence to refute my claim).

Statistical equating of tests is so simple that it should be a crime not to
do it.  All you have to do is decide how you want the test questions to
represent the domain you're testing.  Let's say (hypothetically) that we
want to weight the following areas of juding knowledge in the following
ways:  knowledge of BJCP (10%), knowledge of styles (40%), knowledge of
brewing techniques (25%), and knowledge of flaws (25%).  Now, we categorize
the questions in the exam pool according to which of these areas is
addressed (this has probably already been done).  Next, we go back through
the scores for the past 5 years or so and compute averages for each item in
each area.  Now we have a basis for determining whether tests are roughly
parallel or not by looking at the weighted means for all items on the test.
That way we avoid having tests that are "real boogers" and test that are
overly easy.  Again, the procedure I'm describing is a simplification, but
it gets the point across.

Scott, Denis, whoever, ... can we do this?  Again, I'll volunteer my time
to help.

Ed Wolfe
wolfe at act.org

------------------------------

Date: Mon, 29 Jan 96 13:48:03 EST
From: "George Danz (919) 405-3632" <danz at edasich.rtp.semi.harris.com>
Subject: re: rewarding fat beers

Re: fat beers winning over skinny ones.  I guess I wouldn't have any problem
with rewarding fat beers over skinny ones as long as they meet the following:

1) The style guidelines.  I consider these to essentially spell out what the
        beer should live up to.  If a gravity range of 1.045 to 1.055 is given
        for a particular style then the beer should meet that to be able to
        win in that style catagory.

2) They must truly BE the best of the beers being judged in that class.

My own personal feeling would be to judge a beer best if it meets the 
median numbers for the syle's gravity and bittering ranges.  For example,
provided that a beer has the least flaws of a flight of beers, if it seems
to come closer to sg of 1050 when allowed a range of 1045 to 1055, I would
tend to grade it higher than a beer that either came in at the minimum or
the maximum.

If the beer judging community wishes to widen the guidelines for a particular
style that certainly is fine with me, but aren't the guidelines meant to be
sort of a rule book by which we must abide?  

I think we have to be consistent too.  Just a week or so ago, there was a 
lot of discussion about the BJCP exam not being "tested" or certified for
validity.  Wouldn't it be kind of strange if we 'tightened' up the exam to
make it more valid (I'm not sure I really understand what was being asked
for) to make sure everyone who passed with a certain grade were clones of
one another on the one hand and then on the other hand ignore the style
guidelines when judging?  To me this seems counterproductive.

I suspect that when the BJCP began in history, there weren't as many styles
as there are today.  This fact suggests to me that we solve the problem of
not meeting a particular style by adding another style.  So, if a brewer wants
to win by making an especially heavy Pale Ale, couldn't we add a style,
"Darn Heavy Pale Ale"?  Of course, I suppose we'd be subjected then to more
than "Darn Heavy Pale Ales" as some brewers decided to win the contest by 
submitting a heavier still example of the style, prompting us to create a
"Barley Wine Pale Ale".

Are you guys and gals having as much fun reading this as I'm having 
writing it?
Best Regards,

George E. Danz                    Snail Mail Address:
gdanz at harris.com                  PO Box 13996
(919)405-3632 Work                Research Triangle Park, NC 27709
(919)405-3651 FAX             



------------------------------

Date: Mon, 29 Jan 1996 15:33:01 -0600
From: ddavison at earth.execpc.com (Dennis Davison)
Subject: Rob and Norms Comments

Sir Rob Bates writes:

>There seems to be some misconception in some circles as to what the 
>BJCP is all about.  The charter of the BJCP is now and has always 
>been to test and certify judges. Period.

You are absolutely correct. However, there are a few things that go along
with this. Style Guidelines, that are complete and well formated and other
support services, such as providing lists of judges to organizers.

>Attempts to enlarge the scope of the program may just produce another 
>self-righteous, bloated, bureauratic, holier-than-thou, immovable AHA-
>like entity.

To let everyone know, we had turned down an offer to create a BJCP National
Competition. It was turned down because we don't need that many irons in the
pot. Look at the resources that the AHA throws at the AHA Nationals. Sure
they get paid for this, but in a volunteer organization, we would be asking
a lot of people to give up their lives for a few months.


Norman Dickenson elates:

>I have won
>three best-of-show awards (American Wheat twice with O.G. 1.045 and
>1.047, and a Bohemian Pils with O.G. 1.051) and know or have heard of 
>a number of persons who have won BoS ribbons with low gravity beers

Norm, I have won several firsts and a couple of BoS with a Berliner Weisse. An
OG of 1.032. I would have to ask Bill how this fits the big beer syndrome.
I'm also a judge that knocked a beer for being to big for style. (Actually,
it had to much body for an American Brown). I later found that the beer was
only .004 points above the style limit, but the body made it seem even
higher. Bill, have you ever judged a BoS round. It's not easy. Sure you can
eliminate about 1/2 the beers fairly easily because of some of the more major
flaws. But when you get down to the last 5-10 beers your looking at
difficulty of brewing, freshness or age, balance, and how well it's stood up
over the last hour on the table.

>The CURRENT Exam program has two graders
>for the exam, an Associate Exam Director who review the scores against
>the exam and the Exam Director who also reviews the scores against the
>exam.  All four persons would be knowledgeable of and a participant in
>a process whereby the score assigned to an exam is changed from that which
>was given by the two graders.  At no time does anyone who scores or 
>reviews exams ever know the name of the person who wrote the exam
>until after a score has been recorded by the Program Administrator. 

To take this even one step further, I refuse to grade exams from the
Midwest. Chances are I know some of these people and might recognize the
handwriting. Yes, it's a long shot, but I prefer it that way.

Dennis Davison                                       ddavison at earth.execpc.com

                           BJCC Midwest Region
                         Chairperson of the BJCP


------------------------------

Date: Mon, 29 Jan 1996 17:15:43 -0500 (EST)
From: Scott Bickham <bickham at dave.nrl.navy.mil>
Subject: Here is the BJCP exam

This will be published in the next BJCC newsletter, but this
should answer the general question about what to expect on the
exam.  As I have said, there will be a BJCP question, 4 style
questions, 4 technical questions and a mixed question.  A list
of many, many questions was posted a few years ago, but here
is a sample format for the exam:

1.  Describe the purpose of the Beer Judge Certification Program, how the 
judging levels are organized and how a judge progresses within the program.

2.  Identify, describe and differentiate four common members of the 
<insert category here> family.  Give commercial examples of each style.

3.   Describe, differentiate and compare the taste and aroma characteristics 
of the following beer styles.  Give commercial examples of each style:
	a) Style A	b) Style B	c) Style C

4.  Describe and differentiate the taste and aroma characteristics of: 
Namebrand A, Namebrand B and Namebrand C.  Also discuss their styles.

5.  Identify, describe and give commercial examples of  a major beer style 
associated with:
	a) City A 	b) City B	c) City C

6.  Describe and discuss the following beer characteristics.  How are they 
perceived?  What causes them and how are they avoided and controlled?  Are 
they ever appropriate and if so, where and when?
	a) Flavor A	b) Flavor B	c) Flavor C

7. Describe the hopping schedule for <Style A> and <Style B>, explaining 
why you have selected specific amounts and varieties of hops and how your 
schedule fits the style.

8.   Open ended question on a brewing ingredient or process.

9.  Provide a complete five gallon ALL-GRAIN recipe for <Style>,listing 
ingredients and procedure.  Give original and final gravities.  Explain why 
the recipe fits the style.

10.  Discuss the following brewing techniques.  How do they affect the beer?
	a)  Technique A	  b) Technique B	c) Technique C


Of course, now that this is posted, I will be flamed for having questions 
that do not directly affect one's judging ability, but that's life.
The style questions should be easy for someone who has read Jackson,
and the technical questions can be answered adequately with Papazian,
better with Miller and still better with Noonan and Fix.  Question
6 is useful to measure knowledge of troubleshooting, Questions 7 and 9
are sort of stylistic questions, but from a brewer's point of view,
and Question 10 tests knowledge of the brewing process.  It is helpful
to have an open-ended question on malt, hops, water or yeast to 
let expert brewers show their stuff.

Scott
- -- 
====================================================================
E-Mail: bickham at dave.nrl.navy.mil		FAX:(202) 404-7546
						
Office: Naval Research Laboratories		Home and BJCP:
	Code 6691, Complex Systems		7507 Swan Point Way
	Washington, D.C  20375			Columbia, MD  21045
	(202) 404-8632				(410) 290-7721
====================================================================

------------------------------

Date: Mon, 29 Jan 1996 17:27:10 -0500 (EST)
From: rll at tanis.cptech.com (Robert L. Lamothe)
Subject: Re: rationale for a distribution

	With all this talk going on about the exam, I feel I have to throw
in my $0.02.  While it may be nice to have a standard bell curve for the 
BJCP exam, it isn't realistic.  The concept of a bell curve is based on the
idea that in any cross section of society you will find that certain 
percentages of the group will fall into intellectual groupings based on the
societal norm.  While this may have its place in Primary and Secondary
Education, I don't think it applies well to something like the BJCP.  For
all practical purposes the standard bell curve is used to guage the 
the qualifications of instructor as much if not more than that of the student.  
If you accept the concept of the Bell curve, then any time one does not appear
the instructor must be at fault.  Since there is no instructor in relationship
to a BJCP exam, then the idea of a standard bell curve is flawed.  There is
no base line to work with so no accurate curve can be determined.  Similarly,
my experience with homebrewing suggests that it isn't pursued by an average
cross section of Americans.  All the homebrewers I know are technical or
professional people, with most being among the highly computer literate.

	I think it is a testament to the exam and those who give and take
it that few people fail it.  The fact that through a process of self study
a fair number of individuals each year can score high enough to earn rank
of National or Master suggests the exam is weighted fairly.  The BJCP exam
isn't a competency check to help you get your next job promotion, nor is
it about prestige or to create a glass ceiling.  The exam is about having
the knowledge to examine someones beer and give them inciteful, accurate
feedback as to the attributes and flaws present in the sample.  Grading on
a curve or watering the test would serve little more than allowing 
incompetency to reach higher levels.  There may be ways to improve the
exam and the grading, but I don't believe a bell curve to be one of them.

						-Bob
- -- 
* Robert L. Lamothe                           Corporate Technologies Inc.     *
* Systems Engineer                            100 foot of John st.            *
* (508)459-2420                               Lowell MA 01852                 *
* rll at cptech.com							      *
*                                                                             *
* "All I ask of life is a constant and exaggerated sense of my own            *
*  importance."								      *

------------------------------


End of JudgeNet Digest
************************