To: judge-recipients at synchro.com (JudgeNet Recipients)
From: judge-owner at synchro.com (JudgeNet Administrator)
Reply-To: judge at synchro.com (JudgeNet)
Errors-To: judge-error at synchro.com
Precedence: bulk
Subject: JudgeNet Digest #1201 (Jan 22, 1996)
Date: 22 Jan 96 01:18:21 EST (Mon)
Message-Id: <9601220118.AA07654 at synchro.com>


JudgeNet Digest #1201				   Mon 22 Jan 1996


                               JudgeNet
			 The Beer Judge Digest

                 digest submissions: judge at synchro.com
          administrative requests: judge-request at synchro.com
    send cancellations & rank updates to the administrative address
	  messages sent to the wrong address will be ignored

        WWW Archives: http://www.umich.edu/~spencer/beer/judge

                 Editor: Chuck Cox <chuck at synchro.com>
             Archivist: Spencer Thomas <spencer at umich.edu>

      Publishers: SynchroSystems and the Riverside Garage & Brewery

            Anti-Prohibitionists may also be interested in 
                LiBeerty: The Libertarian Beer Digest
           Subscription info: libeerty-request at synchro.com

     For BJCP General Information contact: geninfo at bjcp.synchro.com


Contents:
  exam question difficulty                 (Wolfe)
  Braggot - Again                          (Fred Hardy)
  testing issues                           (Craig Pepin)
  A Treatise on Mead Judging               (Michael L. Hall)
  exam discussion                          (drbeer)
  Hops are OK in braggots                  (Olson)

----------------------------------------------------------------------

Date: Fri, 19 Jan 96 08:21 CST
From: Wolfe at act-12-po.act.org
Subject: exam question difficulty

Previously, I suggestion that the stability of exam scores on retakes is
not as pertinent as some other factors might be.  Al suggests that a bigger
factor might simply be luck--a person might get questions on a style that
he/she just happens to like a lot and, hence, knows a lot about.  I agree,
but there isn't much we can do about this .

But, another big factor, one that we can do something about, relates to the
relative difficulty of exam items.  For example, I would expect that a
questions that asks an examinee to describe the flavor characteristics of
an Alt to be more difficult to receive full credit on than would a question
that asks the examinee to describe the flavor characteristics of an
American Pale Ale.

There are two reasons why I believe that the description of the Alt is more
difficult than the description of the APA.  First, knowledge of a Alt is
less common and more difficult to find.  The only detailed descriptions
I've seen are from Jackson, Deschner, and the style guidelines.  Every beer
magazine in the U.S. has run some article about the explosion of micros in
the U.S., and hence descriptions of the post popular style associated with
those micros--APAs.  Futhermore, I've never tasted an authentic commercial
example of an Alt.  To my knowledge, there are none available in the U.S..
On the other hand, I'd have a hard time finding a store that sells more
that just megabrews that DOESN'T have at least one APA.

Second, the information that is available about Alts is not in agreement.
Deschner says that the Alts he's had made him sweat hops.  The AHA, until
this year, billed Alt as a malty brew.  (Now they say that there are some
rare examples that are highly hopped.)  The lack of available knowledge and
the general lack of agreement between experts on the Alt style make
answering questions like this more difficult for the examinee than would be
answering questions about APAs--a style that nearly every homebrewer has
tasted and brewed.

This problem of relative difficulty disadvantages the examinee that
receives the Alt question in two ways.  First, it is more difficult for the
examinee to get full credit for the question because the information is
more difficult to "master."  Second, it is more difficult for the graders
to agree upon a score because the information about the style is available
is difficult to agree upon.

My suggestion is to examine potential problems like this and identify ways
to deal with common scaling problems like the example I've given.  The
techniques for doing so are very simple and inexpensive.  By doing so, we
can be sure the the exam scores are similar regardless of who grades the
exam and what questions the examinee is asked.  I hate to sound like a
broken record, but we can't be sure that these are not important issues
unless we take a careful look at examinee scores to evaluate the validity,
reliability, and fairness of the certification exam.

Ed Wolfe
wolfe at act.org

------------------------------

Date: Fri, 19 Jan 1996 11:16:03 -0500 (EST)
From: Fred Hardy <fcmbh at access.digex.net>
Subject: Braggot - Again


At least Brews agrees with me. That's scary!

Hops should never be in braggot. If they are in mead, then they are an 
herb, and the beverage is correctly called a metheglin. If they are in a 
malted grain beverage, it's beer.

As for carbonation, why not? We know that the Celts and Brits were 
skilled coopers before the Roman invasion of England. A braggot, beer or 
ale stored in casks very well may have been carbonated. I believe braggot 
guidelines should provide for still and sparkling braggot. The 
carbonation level should be the same as a real ale - never fizzy. Priming 
with about 2/3 cup sucrose should do it nicely for a 5 gal. batch.

I have entered still braggot in competition, and had one judge comment 
that it would score better if it was carbonated. Please, judges, this is 
braggot, not beer. I personally like still braggot.

The article by Dr. Michael L. Hall (as Brews points out) proposes a 
classification of braggots into pale, amber and dark. He describes these 
from his concept that each would be made using very light honey and all 
color is from the types of malt used, IHO. Calling for a roasted malt 
flavor and aroma in dark braggot is, IMO, silly. Using 80L caramel, 
Special B and dark wildflower honey makes a hell of a dark braggot with 
no noticeable roasted malt flavor.

My "King Arthur's Own" braggot is made from English mild and home made 
brown and amber malts. Wildflower honey is also used. The result is quite 
dark with none of the characteristics Dr. Hall proposes. The point is, 
color is not a good basis for classifying braggot or defining flavor and 
aroma profiles. Beyond still or sparkling, there are few (no?) variables 
which could be used to discriminate among braggot subcategories.

Braggot should have definite malt AND honey character. Either may 
dominate, but neither should. Young braggot (3-6 months) will usually 
have a note of sourness and obvious alcoholic sharpness which 
dissappears with aging. During the Middle Ages, however, old braggot 
brought the highest price. It would have been very smooth, amber to 
opaque, and may have been sparkling. Since it has not been commercially 
availble since the Middle Ages, why should we expect it to have different 
characteristics today?

Notice, I don't claim that braggot can not have roasted malt character, 
only that it doesn't need that simply because it's dark. Like beer, 
the possible combinations of malts and honeys available today allows for 
an infinite range of colors, aromas, flavors and enjoyment. Let the best 
braggot win!

Cheers, Fred

==============================================================================
We must invent the future, else it will  |          <Fred Hardy>
happen to us and we will not like it.    |
[Stafford Beer, "Platform for Change"]   |    email: fcmbh at access.digex.net


------------------------------

Date: Fri, 19 Jan 1996 12:30:15 -0500 (EST)
From: Craig Pepin <ckp at acpub.duke.edu>
Subject: testing issues

Ed Wolfe raises some very valuable points about the BJCP exam, points that call
for further discussion even if I remain skeptical that we can ever reach a test
that is truly "valid" and/or "accurate" (whatever those ambiguous terms really
mean) without going to an exclusively multiple choice format with no tasting.
That would eliminate all excess variables in grading but also is an option I
don't think anyone wants to seriously explore. (although I do think a small
component of the test could be changed to such questions).  Judging is an
inherently subjective discipline, and although we should strive for greater
objectivity in our testing means, I don't think complete "objectivity" or
"validity" is ever  achievable except at too great a cost to the essence 
of the discipline itself.

Interrater reliability is an area where I think the BJCP can be improved, and is
currently being improved as evidenced by Scott's work on the grading guide.
Perhaps the only way to truly establish this with any social scientific degree
of "validity" is to make the grading guide several hundred pages long,
consisting of a "perfect" answer for each of the 100 questions that now make up
the exam pool.   Even then, there will be varying interpretations of graders as
to how well an answer measures up to the given ideal (is it a 7 out of 10, an 8
out of 10)?  And will a person whom memorizes word for word every "perfect"
answer and then regurgitates this on the test be entitled to a perfect score of
100?  (Maybe there should be a special "master, anal retentive" rank for such a
hypothetical tester.)  One way this is addressed in my field (Duke U. Dept of
History) in a large lecture course is to have the same grader grade one portion
of the exam, or a subset of the questions asked.  In a block of 200 exams, the
students can therefore be assured of interrater reliability on each subset of
questions.  This works if the graders live in close proximity and meet on a
regular basis.  I would never dream of creating such a logistical nightmare for
Scott, however.  But Ed is right - perhaps there are reasonable ways that this
can be improved, *without* driving away the graders who already volunteer a lot
of their time for no direct compensation other than a small fee and the good of
the program.

On the question of alternate forms reliability, I think this is an issue, but 
not one I can see any kind of solution for.  Every test carries with it some
subjectivity with regards to content, it seems to me:  Unless you test for
the entire body of knowledge a taker is expected to master (How about a 30 hour
test with all 100 questions in the pool?) there is necessarily going to be a
small element of luck, as Al Korzonas pointed out, although anyone who prepares
with a broad-ranging knowledge over all styles and fields is more likely to do
well, and will most likely make a better judge anyways.

I'm not sure what is meant by internal consistency (stability within a test)
 - simply that the questions do not require duplicating answers?  This is an
issue which could probably be addressed very easily.  A simple awareness of this
when the questions are selected for each exam should be sufficient.

Ed, I'd very much like to hear some brief descriptions of ways in which we might
test the BJCP exam in these various areas.  I think it would be a valuable
exercise if properly applied.  It sounds like you have a lot of expertise about
principles of testing and measurement.

Craig Pepin
Triangle Unabashed Homebrewers
Durham, NC

------------------------------

Date: Fri, 19 Jan 96 11:13:31 MST
From: hall at galt.c3.lanl.gov (Michael L. Hall)
Subject: A Treatise on Mead Judging

Mead Enthusiasts and Judges,

I recently published an article in the January 96 issue of _Inside Mead_,
the publication of the American Mead Association. The article concerns
mead judging, and includes a description of a mead competition which can
be scaled to accommodate both small and large competitions. 

While I encourage all of you to support the AMA by buying _Inside Mead_,
I have also decided to make the article available on the net. The
article is available in PostScript and ASCII text form on the Los Alamos
Atom Mashers "Goodies Page" at:

    http://alpha.rollanet.org/~tamhc/goodies.html

If possible, please use the PostScript version because its formatting is
closer to the way I want it to be.

To whet your appetites and let you decide whether or not you're interested
in the article, I am including the outline for the article at the end of 
this posting.

Fermently,
Mike Hall

- ---------------------------------------

Outline - Mead Judging
- ----------------------

Preamble
  Intro to Mead Judging
  Problems:
    - show mead
    - cyser and pyment, hippocras categories usually under-represented
    - varietal honeys sometimes get short-changed
    - delicate meads get overlooked
  Predispositions:
    - honey should be expressed
    - wine character not appropriate, except in pyments

Treatise
  Introduction
  Categories
    - Modifiers
      + Varietal
      + Strength: Hydromel / Standard / Sack
      + Sweetness: Dry / Medium / Sweet
      + Carbonation Level: Still / Sparkling
    - Traditional Mead (Standard Traditional Mead, Varietal Honey Traditional
                        Mead)
    - Melomel (Cyser, Pyment, Raspberry Melomel, Cherry Melomel, Plum Melomel,
               Peach Melomel, Apricot Melomel, Strawberry Melomel, Blueberry
               Melomel, Berry Melomel, Citrus Melomel, Tropical Fruit Melomel,
               Prickly Pear Mead, Other)
    - Metheglin (Mulling Spice Metheglin, Beer Spice Metheglin, Mellow Spice 
                 Metheglin, Italian Spice Metheglin, Flower Petal Metheglin,
                 Peppery Metheglin, Chile Mead, Other)
    - Braggot (Pale Braggot, Amber Braggot, Dark Braggot)
    - Mixed Category Mead (Hippocras, Apple Pie Mead, Spiced Melomel or 
                           Fruited Metheglin, Maltomel, Malteglin, Other)
  Instructions for Organizers:
    - Procedure (basically same as beer judging)
    - Entry form 
    - Stewarding concerns 
    - Smaller flights 
    - Categories 
  Instructions for Judges:
    - before the judging
    - judging a sample mead
      + overall concerns: balance (acid-sweet-tannic, honey-other),
                          expression of honey, expression of other
                          ingredients
      + procedure (aromas, appearance, tastes)
      + off-tastes (nutrient tastes, phenolics, higher alcohols - harshness,
                    oxidation - not bad in cherry/plum melomels, skunkiness - 
                    should be none, metallic)
    - judge silently
    - lots of comments
  References

Author bio

------------------------------

Date: Fri, 19 Jan 1996 23:29:55 -0500
From: drbeer at tiac.net (drbeer)
Subject: exam discussion


Al K. writes 

>My point is simply this:  part of your score is simply luck that the styles
mentioned in the exam are your favorites.


I have graded many exams and I think you miss the point.  You indicate you
are more familiar with some styles than others.  In order to do well on the
exam, i.e. achieve Master score levels, you should know all the material
thoroughly.  While exam question selection may give you a slight advantage
in an area in which you are more familiar my experience has been that a
significant enough range of styles are covered on any given test so that a
grader can tell when you have a good breadth and depth of knowledge as
opposed to only understanding certain areas.

Since the duration of the exam is finite from a practical standpoint only so
many questions can be asked.  We could of course reconsider and make the
test last several days like medical or architecture exams, with several
parts (technical knowledge, style knowledge, ingredients, program knowledge,
tasting, etc.) and you would only have to pass each part once.  However I'm
sure very few people would continue to be interested in such a test.
Personally having graded lots of tests and taken it twice myself I believe
(and yes there is no empirical justification for this belief) that the test
as presently stands does a pretty good job of ascertaining a persons
capabilities within the four coarse brackets.

Mostly I view the test as a gating factor.  It determines whether a person
has enough knowledge to begin in the program, and how far  that person
should be able to progress before demonstrating that they have improved
their knowledge to compensate for previous shortcomings.  As such I view the
test as more of a proof of basic competency and recognize that even those
who score high and have participated in the program and accumulated many
experience points still always have something to learn.


                          Jay 

- ---------------------------------------------------------------------------
Hopfen und Malz, Gott erhalt's

This is a key free document, no keyboards were harmed in its creation.

If Bill Gates is such a visionary how come he couldn't envision that
computers would have over 1MB over memory when he began selling MS-DOS over
a decade ago??


------------------------------

Date: Sun, 21 Jan 96 22:06:30 MST
From: olson99 at mack.Rt66.com (Olson)
Subject: Hops are OK in braggots

In JudgeNet#1198 Fred Hardy argues against allowing hops to be used 
in braggot:

>Beer is an alcoholic beverage made from malted grain(s). It
>usually has hops in it, but may not. If the brewer says it's
>beer, I agree. If the beverage is made from malted grain(s) and
>contains hops and honey it is a specialty beer according to AHA
>guidelines. If the honey comprises 50% or more of the fermentable
>sugars it is specialty beer with a lot of honey in it.

Sorry, I have to agree with the AHA rules. They say that if more than 50%
of the fermentables are honey, it is a mead, ie. a braggot, not a specialty
beer. There may be small competitions out there with other rules. But I 
agree with the AHA on this very simple test between beers and meads.

>Braggot is made from malted grain(s) and honey, but should never
>contain hops (otherwise it's beer). The AHA guidelines for
>braggot show "0" (like in ZERO) IBUs under the bitterness column.
>I urge judges to consider this if they are fortunate enough to
>judge braggot. It is not beer, and the absence of hops gives it a
>character very unlike beer.

Sorry, but having hops does not suddenly make it a beer!

Last weekend I judged at the Ambrosia Adventure in Denver, a mead only
competition. One of the metheglins I judged was spiced with cascade hops
and spearmint. Not a combination that I would have thought of, but it
was interesting and well made. Was it a beer? Of course not, it had
no malt.

The winning braggot was a rauch braggot. It has a nice smoky aroma and
taste along with good honey character. Whether it had hops or not was
irrelevant.

>Having endorsed one AHA guideline for braggot, let me knock
>another. Color is given as 0 to 5 degrees L (SRM). This is, at
>best, simply wrong. Braggot made with the lightest of ingredients
>will be at least 4 degrees L, and that would be a very
>untraditional braggot. The most definitive guideline for braggot
>color is: "color may vary widely depending upon ingredients."

As we all know, the AHA guidelines are simply that, guidelines.
They are not hard and fast rules. Just as a wide range of colors
should be allowed, a wide range of hop bitterness should be allowed,
from zero to a large IBU number.

>I base both comments on the fact that braggot essentially
>disappeared from the end of 15th century until the recent rise of
>homebrewing. Prior to the 15th century this British (and perhaps
>Norse) beverage would have been made with blended (wild) honey
>and home kilned malt. Neither of these was likely to have been
>light, and braggot was probably deep amber to brown.

>During this period, hops were not widely used in British brewing,
>and did not become totally accepted until around 1700. Hops in
>braggot was possible, but highly unlikely. What is likely, but
>which I have never seen addressed, is the presence of spices in
>braggot. The guidelines should be expanded to clearly allow
>spices, since they certainly were in most British braggots made
>during the Middle Ages (450 AD - 1400 AD).

Why should we narrowly define braggots in terms of one style of drink
made in one location during one time period? When hops were introduced
into brewing in Europe, there was a period of many years when hops were
legally not allowed in England and English ales. If someone were to now
to insist that the only true "ale" is one that has no hops, I would
strongly protest. We no longer live in those times.

Fred, I respect your interest and knowledge about historical brewing, but
please don't go over board and insist that they some how wrote the book on
braggots and then closed it to any future changes.

Purists have put a "show" catagory into some mead competitions. In this
category acid blend and other adjuncts are not allowed. If you wish to
propose a subcategory of braggots that are only honey and malt, please
go ahead and propose it. But please do not try to disallow the use of
hops in the whole category.

I truly do not understand why you are willing to allow spices
in braggots, but not hops!

Sorry for this long post, but I believe this is an important issue that
all judges, not just mead judges, should be aware of.

Gordon Olson
BJCP National Judge
Maker of beers and meads for 15 years.


------------------------------


End of JudgeNet Digest
************************