Beer Judging Methods - Something not quite right?

Post Reply

Topic author
PistolPatch
Gold
Gold
Australia
Posts: 5286
Joined: 7 years ago
Location: Perth, Western Australia
Region: Oceania
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City:

Beer Judging Methods - Something not quite right?

Post by PistolPatch » 3 years ago

Am just approaching Section J of the BIABacus help.

Section J is based on the BJCP beer appreciation method. Other organisations have similiar structures. None of them agree (of course, like every other measurement in brewing :roll:) but the basics they rely on are the same and I think they are wrong [EDIT: I think I am wrong now :) - see post #5]. Let's look at Section J of the pre-release BIABacus...
1.0 - Section J.jpg
Forget what judging system you use, whether the beer is being judged out of 20 or 50 points in total, they all fix a proportion of those points to a certain beer quality.

In the BJCP system for example, they assign 12 out of 50 points to aroma. I asked a BJCP judge what happens in a beer style where there is little or no aroma. He replied that you just give it 12 points. I have a problem with that straight away. (I know the opposite can be argued but I think that is totally lazy.) [EDIT: Once again, see post #5].

Here's some more thoughts. If you are going to have categories, I can see a lot missing. Can you?

Why wouldn't one major category be balance? Another category could be complexity. Another one could be subtlety. I don't know. But one thing I do know is that aroma and flavour in something like an APA are major things I enjoy whereas in a lager, a very subtle complexity of hops and malt is what I am looking for.

So what I am saying is that, perhaps we should stop and consider each style individually. I'd go as far to say that some styles should be judged on say four categories whereas others might need eight. This would be an inconvenient truth but it could well be the truth :scratch:.

What do you think? Don't be afraid of speaking up. We have discovered heaps of other dodgy info out there so why not this one as well?

Anyway, I'm drunk now and going to bed :lol:.

:)
PP
You do not have the required permissions to view the files attached to this post.
Last edited by PistolPatch on 27 Jun 2014, 00:12, edited 1 time in total.
If you have found the above or anything else of value on BIABrewer.info, consider supporting us by getting some BIPs!


Contrarian
Gold
Gold
Posts: 228
Joined: 4 years ago
Region: Please select one...
Preferred Brewing Method: Please select one from below...
City:

Beer Judging Methods - Something not quite right?

Post by Contrarian » 3 years ago

From my very limited perspective there are always going to be huge inherent problems in trying to quantify something that is as subjective as taste. The best you can hope for is as system that allows for good inter rater reliability against certain criteria. So for a competition you want all the judges to be scoring equally against a standard that is pre agreed and allows consistency between different locations at different times.

For a home brewer or beer lover the main art is in learning how to describe flavours and characteristics of beer that are desirable to an individual (or not) so these can be targeted when making or purchasing beer. For example do I like passion fruit hop aroma or not.

In a very long winded way what I'm trying to say is that whether BJCP style guides or any other judging parameters are correct or not they are a generally agreed standard. I'm not sure that introducing another version would do anything other than muddy the waters even further.

Unless you are specifically trying to brew to style the most important question is do I like this beer and if so why.

User avatar

Lumpy5oh
Gold
Gold
Canada
Posts: 423
Joined: 3 years ago
Location: ON
Region: USA & Canada
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City: Woodstock

Post by Lumpy5oh » 3 years ago

I also think that adding more categories would "muddy the water." My take on the scorecard is that a judge would know what style of beer is being judged and what characteristics it should have and score each category accordinly.For example; a beer that should be complex and is would score higher in the flavor category than a beer that isn't as complex as the style warrants. On the flip side; a beer style that is supposed to be suttle would score low in the flavor category if it is slightly overpowering.
I would think that things like balance and complexity would be thought of in the flavour and aroma categories and would score well if in line with the particular style.
Some people are like slinkies. Not good for much, but bring a smile to your face when pushed down the stairs.

Weehoosebrewing.ga

User avatar

Rick
Gold
Gold
United States of America
Posts: 760
Joined: 4 years ago
Location: Pennsylvania
Region: USA & Canada
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City: East Greenville

Post by Rick » 3 years ago

I'm very unfamiliar with the BJCP methods of scoring. One day want to become formally educated to better understand what this system is all about.

Aroma points deserve a bit more attention, that's rather ridiculous!?

Semi-related: I really want one of those sensory training kits, but they are quite spendy.


Topic author
PistolPatch
Gold
Gold
Australia
Posts: 5286
Joined: 7 years ago
Location: Perth, Western Australia
Region: Oceania
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City:

Post by PistolPatch » 3 years ago

Thanks for having a think on this especially as my OP was very muddy. Got rid of some of the stream of consciousness stuff in the OP. Your posts above have definitely helped me get a bit closer to what is bugging me (and also to remember that there is a bit more sense in the judging system than initially appears. More on that last bit at the end).

I think there are two different issues. Judging for judges and beer appreciation for the individual. The latter is much more important to us but let's work our way there....

Judging

1. The BJCP and other judging systems have had some of the best palates work on those systems and in the next section, we'll see that there is actually a lot of sense in the system (see below).

2. I think that the system they have might be okay for those who want to get fully trained into it and who are fortunate enough to get an excellent teacher. But I worry they are not that great for others wanting to learn about beer appreciation.

3. Existing different judging systems give different weights to different characteristics and, in fact, there is not even a standard agreement between countries on what beer styles are called and defined. (Typical of the brewing game. Look at malt names, Colour and IBU estimation formulas and definitions, grain spec descriptors etc etc. :roll:).

A Bit More Work

1. If you look at the BJCP style guidelines or any other, they are often not that comprehensive, In fact, many style guidelines are very brief and very vague.

2. But then you get a nice bit of work like this Beer Scoresheet***. (You'll see in that lumpy how many attributes are lumped 'underneath' the five main characteristics. This reminded me that the aroma thing actually makes some sense.)

Could lead to education for everyone...

Imagine that each style not only had far less vague descriptions but also links to excellent articles on that particular style? Imagine if each style had something like the score sheet above completed for each style? Some sort of sheet where you could visually see that such a style should have this trait (tick) but not another (cross)?

That would be pretty good. And that is what each of you have asked for above - a good way to learn about beer, describe it and know why you like it. It's what I have wanted for ages as well.

The right tools and right teacher...

I paid to do a BJCP course here about four years ago. I was really excited about it as a guy called John Jens (great guy and totally passionate), taught me about wine around twenty years ago. Two hours a week over six weeks and at the end, he gave us a wine and we all had to identify not only the variety but also the region and then part of that region the wine came from. From memory, I think nearly all of us got everything correct. It was an amazing experience.

So I roll up to the BJCP course and was really disappointed. Not only is a lot of the course studying to learn things by rote but you were not educated (lead forth) as John did to us. John would give is a wine and then not say, "What do you smell?" He would say, "Can you smell this?" That is a major difference when educating someone in this area, at least for someone like me who does not have a natural knack at it.

Look at the first thing on that BeerScoresheet - 'Grainy' under aroma. I still to this day have had no one say to me, "Smell this beer. It is really grainy." I actually need someone to lead me through each of those aspects, one by one."

I think you need a great teacher as beer really does not travel well in many situations. Most of the beers we had on that course (which I stopped about half-way through) were bloody awful. APA's that had been transported badly from America and therefore had no hop flavour or aroma left. Or perhaps a European lager, once again transported poorly, old and skunked.

Now none of us might get good access to a great teacher but I can't see any reason why the tools shouldn't be made easy for us. It just requires someone with the skill putting in the time and effort for us.

...

As to what to do in Section J of the BIABacus, I don't know but I better stop writing here and keep on with the other help. Thanks for the above and getting me focussed again. If you have any ideas of how Section J can be used to lead people in the right direction, please post away.

:peace:
PP

*** Never had that sheet in my course. Nothing like it at all.
Last edited by PistolPatch on 28 Jun 2014, 21:12, edited 1 time in total.
If you have found the above or anything else of value on BIABrewer.info, consider supporting us by getting some BIPs!


Yettiman
Gold
Gold
Great Britain
Posts: 188
Joined: 4 years ago
Location: Sicily
Region: Europe
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City: Palermo

Post by Yettiman » 3 years ago

Hi,

Firstly let me qualify my thoughts by saying I do not regard my self as a homebrewer yet, but rather a dabbler in homebrew, only having 12 home brews under my belt. But I have joined a kcal brewclub where there are two homebrew national judges (UK) and I have stewarded at National level (again UK) so have been privileged to listen in on some informative conversations.

The thing that stuck me was the reasons for the classifications and they were laid out was to try and reduce he amount of subjective marking and to also advise the competitors how close they were to the agreed standard.

The marks awarded (read comments) had nothing to do with personal preference or how good the beer was but rather, "Was it the right colour?, Did it match the standards level of acidity?"

If we stry too far from the Black & White and into the Grey subjective, it will be very hard to get any sort of agreement on the results.

I hope thus makes sense (sorry if badly written), just trying to add in my thoughts

User avatar

Lumpy5oh
Gold
Gold
Canada
Posts: 423
Joined: 3 years ago
Location: ON
Region: USA & Canada
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City: Woodstock

Post by Lumpy5oh » 3 years ago

PP
That beer score sheet is exactly what I was thinking of.
I wholeheartedly agree that someone would need proper training to use the sheet successfully. Without proper training I couldn't identify the flaws. It's unfortunate that the BJCP training you received was so poor. The system can't work without proper training. I cringe to think of how many people receive that level of information and how many of those would go out into the world believing that they are truly certified.
Some people are like slinkies. Not good for much, but bring a smile to your face when pushed down the stairs.

Weehoosebrewing.ga


Topic author
PistolPatch
Gold
Gold
Australia
Posts: 5286
Joined: 7 years ago
Location: Perth, Western Australia
Region: Oceania
Preferred Brewing Method: Single-Vessel All-Grain (SVA)
City:

Post by PistolPatch » 3 years ago

Yetti, yep, I think what we all want is to make this more black and white and less grey. At the moment I thihnk there is too much grey.

The scoresheet that Lumpy mentioned is a big step forward but I spent some time tody on it and can still see many grey areas which is great because I also saw a lot of other possibilities to make this beer appreciation (and maybe judging) area really easy. I can't do the stuff I envision with the BIABacus in spreadsheet form so we'll just have to hope we find funds/resources to code it.

As for the BJCP course I started, most of it then (maybe 3 years ago?) had a lot of rote learning. For example one exam question is, "Describe the characteristics of this beer style."

Personally I think that is a pretty stupid question as it means you have to learn every one of the billion styles for a start. There really aren't too many people in the world that would be able to have ever tasted/experienced all styles. I don't think asking them to learn the style notes by rote is an answer to that problem.

Anyway, putting that aside, the guys on my course had the confidence and knack to be able to describe the beers. Often they all said something different but this was said to be a good example of palate calibration as Yettiman mentioned (but I really think is a furphy ;). But also, nearly all those brewers, one on one, if I talked with them, I really respected their opinion as they were very honest and unassuming...

In fat, the qualified judges I personally know all admit that judging is super hard. Tasting one beer after another and having only a few minutes to write anything is really a pretty ridiculous scenario and the amateur section is given far less resources than the pro brewer judging. It takes more than two or three minutes to judge a beer but they are rarely given that time for amateur entries.

I also have tasted some judges beers and found faults they can't taste and vice versa. So, it is all interesting.

What I find really interesting though is getting back to my earlier post here on wine. When we had a good teacher, we all came up with the same answers. Sort of expected this in my BJCP course but there was not a leader there to say, "Focus on this aspect." (And the beers were all crap anyway as I already mentioned - lol!)

Anyway, I am unhappy with Section J help but no other software even addresses that area so I probably shouldn't be too worried. For now, it will have to stay pretty basic.

:peace:
PP
If you have found the above or anything else of value on BIABrewer.info, consider supporting us by getting some BIPs!


Contrarian
Gold
Gold
Posts: 228
Joined: 4 years ago
Region: Please select one...
Preferred Brewing Method: Please select one from below...
City:

Beer Judging Methods - Something not quite right?

Post by Contrarian » 3 years ago

Just had a quick google and the only beer tasting courses I could see in Sydney weren't running at the moment.

I have also completed a good wine tasting course which ran for a full day. From the outset the trainer stayed he wasn't planning on turning people into wine wankers but wanted everyone to be able to identify characteristics they enjoyed in wine so they were able to buy wine that they wanted to drink.

With the emergence of craft beer I'm surprised there is no beer equivalent. I'm sure all the hipsters would be happy to fork out there hard earned for a greater appreciation of beer. Maybe there's an untapped market here just waiting to be filled.

I have problems with my own beers at times when I am not completely happy with the outcome but have no idea whether it is a process flaw, undesirable hop combination or something else. While I have the rest of my life to work this out through trial and error I would happily pay for a course in beer appreciation that helped me identify the styles I enjoy drinking and how to brew them more consistently.

Post Reply

Return to “Beer Appreciation (incl. Fault Identification)”

Brewers Online

Brewers browsing this forum: No members and 1 guest