Zachary Houghton resigns as an ENnies judge

[El Mahdi]

Well, all the staff members that weighed in seemed to address every point of Zachs explanation in a very logical and believable way. After hearing their side of things I really do believe them. I just don't understand Zachs letter in light of this. He seems very genuinely troubled by the things he feels are going on. It sounds like there is quite a bit of disagreement between some ENnies staff members, and between the ENnies staff and publishers/internet product providers. It sounds like you guys have some internal things to deal with, and a bit of a PR problem also. But, I definitely appreciated all of the staff members who weighed in to clear things up, despite the fact this seems to be a very emotional and troubling issue for you guys. Whether some people are happy with the things you guys said or not, I feel your "transparency" has definitely increased.

Good Luck guys.

 

[roguerouge]

Okay, Meghan, I've read your posts here and on your thread on brilliantgameologists. I've read several long posts from you and waded through your board and I still don't see the big deal.

So. Your evidence thus far:

* Judges on the take? Nope. Not only that, but emphatically nope.

* Fees "unprecedented"! So... how much are the fees? Why don't you say? First off, talk to me when you have to buy thousands of dollars worth of school supplies for your high school classroom or travel to academic conferences without compensation or be left off next year's adjunct teaching list. Second, there's no fee, so that's strike two for you.

Together, these mistakes shot your credibility. Thus, these objections arose:

* You write: "the one who won, Changeling the Lost, had some of the writing I've ever seen in a book." Okay. Why? Can you give me some examples? Can you compare and contrast with other selections? Do you have any evidence other than that you say so? Let's see what your link says: "Changeling: The Lost: Idiot, Full of , and . Rating: 0" I see. So, still no examples.

* A judge on his blog stated that he liked a podcast. And it got nominated. A judge really liked one of the few works on Epic level play. And it got nominated. But what you don't prove is that one causes the other. See this common logical fallacy for what I suspect is going on here, because you undermined your credibility.

* You write: "Animalcast has a lot wrong with it and just isn't nearly the quality of many other podcasts- audio or content. A true critical, objective analysis would not have placed it in the top of all submissions." Again, can you give examples? Will you provide evidence to back up your assertions? Is that really too much to ask?

* You write: "And then another book which we were just confounded by as a nomination- Epic Role Playing- wasn't in the same class as the other nominees. It seemed like a good start, but very amateurish." How so? Examples? Any piece of evidence beyond your unsupported opinion?

 

[roguerouge]

Which leaves you with Monte's work being let in despite being released 6 days late.

Which I, as a consumer of the game but nothing more than that, could care less about.

 

[Meghan]

Okay, Meghan, I've read your posts here and on your thread on brilliantgameologists. I've read several long posts from you and waded through your board and I still don't see the big deal.

So. Your evidence thus far:

* Judges on the take? Nope. Not only that, but emphatically nope.

* Fees "unprecedented"! So... how much are the fees? Why don't you say? First off, talk to me when you have to buy thousands of dollars worth of school supplies for your high school classroom or travel to academic conferences without compensation or be left off next year's adjunct teaching list. Second, there's no fee, so that's strike two for you.

Together, these mistakes shot your credibility. Thus, these objections arose:

Ummm... ok. I never said either of these things actually. I said the point about money was moot because when I entered the conversation it had already been resolved. So- uh- good to know my credibility was damaged over something I didn't say. EDIT-- OH! I see where you got that now-- uh, you do realize that that was a quote from Zachary, right? I didn't actually write that-- hence the "From Zachary the First" with a link to where he posted that?

* You write: "the one who won, Changeling the Lost, had some of the writing I've ever seen in a book." Okay. Why? Can you give me some examples? Can you compare and contrast with other selections? Do you have any evidence other than that you say so? Let's see what your link says: "Changeling: The Lost: Idiot, Full of , and . Rating: 0" I see. So, still no examples.

Uh- ok. Yeah, that's not a blog. That's a podcast. You have to listen to it to get the content. What you are reading are show notes.

And yes, there are specific examples in that episode to substantiate that claim. There are specific entries that I writing samples I cite from each book in the Best Writing category that show example of "good" versus "poor" prose. As in what an English teacher would accept. Each entry was compared and contrasted.

* A judge on his blog stated that he liked a podcast. And it got nominated. A judge really liked one of the few works on Epic level play. And it got nominated. But what you don't prove is that one causes the other. See this common logical fallacy for what I suspect is going on here, because you undermined your credibility.

A judge stated- before submissions- that he liked a specific podcast. And then on the back of one of the books (Epic RPG- not "Epic level play") is a quote by him saying how much he loved this game. And they both ended up nominated.

And I don't want it to seem like I'm picking on Zachary- he was just the most transparent of the judges. Yes, I saw what some others had, but he was the most visible face so the easiest to get direct quotes from.

If a board of a nonprofit was choosing a new landscaper and one of the board members was cousins with one of the landscapers, he would have to excuse himself from the vote. If someone was judging a play competition and had written a testimonial on how much they loved one of the competitors beforehand, they would excuse themselves from the judging process.

To argue against any claim of objectivity further demeans the process.

* You write: "Animalcast has a lot wrong with it and just isn't nearly the quality of many other podcasts- audio or content. A true critical, objective analysis would not have placed it in the top of all submissions." Again, can you give examples? Will you provide evidence to back up your assertions? Is that really too much to ask?

No- we did not review the "Fan Media' category or whatever it was called. This is the only case where there is not evidence to back up my assertions other than the product itself. I did not do a true critical objective analysis of the podcast- but I'm working on it for this one as well as others, so stay tuned.

* You write: "And then another book which we were just confounded by as a nomination- Epic Role Playing- wasn't in the same class as the other nominees. It seemed like a good start, but very amateurish." How so? Examples? Any piece of evidence beyond your unsupported opinion?

Again- I had linked along with this statement the episode which did review the product. There was no unsupported opinion made in our singular ENnies review episode.

But that's just 1 episode of 1 podcast in 1 genre. My point in pointing out our process of review is that if we- a "fan" endeavor can work to be objective, why can't an award system?

No one is perfect. Nothing is perfect. There are varying degrees of "wrong" everywhere. It's the publishers who keep up to date and change based on the market who succeed. It's the podcasts who listen to feedback and take the parts that apply and let the rest go who improve. An ineligible product made it through without notice (yes, I've read the official statement, but its still a pretty hefty mistake). A respected judge resigned claiming "I am disillusioned with the purposeful lack of transparency in the awards" (yes, I read that official response as well). Many podcasts boycotted this year because of the drama caused by last year's podcast entry and forced resignation of an entrant.

This is feedback. The process should change. It needs to change. It can enact changes within the mission of the award that do not disrupt the system too much but will lead to much greater success and reduce these feelings of ill-will towards the award. I still believe that the ENnies are a good thing. My main thought of them is getting to go to the ENnies with Monte and Sue Cook as a gift from Diaglo's winning the date with them. It was a blast and a great time and I provide the feedback in the interest of something I like reducing the controversy and drama.

 

[Thunderfoot]

ENnies Staff Member and not a Judge - oh how glad I am...

Another side of the coin here so that some perspective may be shown on the work involved. I produce the audio for the awards, this has grown from being placed in front of a CD player with a handful of CDs to this year me going into my studio, editing cuts and splicing an entrance and intermission soundtrack. How much did I get paid for this month and a half of work? Nothing - know how much it costs me to do that? At $50 an hour of studio time... Thankfully I have some ins (and a home studio to edit with) so I get a VERY good rate, but can you imagine if I couldn't?

Why do I do it you ask? Prestige - hardly, I barely show on the staff radar. Compensation - obviously not, I don't get paid and don't even get the free product of the judges. Countless groupies and fans - BWAHAHAHAHA! I do it for love of the hobby, just like the podcasters, and webheads - it's why we all do it. Trust me, if you had been in on the meetings a week or two before GenCon you would realize that a lot of stuff goes on behind the screen that has to happen in order to make it work. So why don't we tell you what goes on behind the screen, because it doesn't matter, it's logistics. Unless it ACTUALLY affects the awards, it doesn't go public (or at least it SHOULDN'T).

Once the nominations happen it's up to the voters - the judges and staff have no say after this (other then the judge's award but that's a separate category). I personally didn't agree with MOST of the winners (won't post my opinion here), but that's what you the voting public decided. What upsets me is that Zach emotionally puked on the public before he aired his problems with the other judges - that is just unprofessional.

As for the amount of podcasters that entered - yes there were a bunch, most of them didn't follow procedure and were tossed out, leaving less than 10. Were some mistakes made (specifically the Malhavoc issue)- yes, of course, this is done in the spare time of the folks involved. Everyone tries to do their absolute best to ensure perfection, but things get through. If we had a paid staff to go over all of this with a fine toothed comb it would be different, but we are all volunteers. We do what we can and soldier on. If you want to pay me to do this full time - hey, I'm all for it.

 

[roguerouge]

"Exhibit #1) From Zachary the First:"

When you quote something someone else says and label it like that, you make it your argument too. You become responsible for its accuracy. That goes double when you endorse its views with the heading, "The ENnies are crap." So, yes, it does undermine your credibility.

And, frankly, I'm not going to listen through a 78 minute episode to find the part where you provide the evidence that you should be citing in one of your lengthy posts on the subject in two forums. If you want to persuade audiences of the rightness of your opinions, you need to make it a bit easier for them to get at the meat of your argument than that. What. Is. The. Evidence. You. Cite. There?

 

[roguerouge]

A judge stated- before submissions- that he liked a specific podcast. And then on the back of one of the books (Epic RPG- not "Epic level play") is a quote by him saying how much he loved this game. And they both ended up nominated.

And I don't want it to seem like I'm picking on Zachary- he was just the most transparent of the judges. Yes, I saw what some others had, but he was the most visible face so the easiest to get direct quotes from.

If a board of a nonprofit was choosing a new landscaper and one of the board members was cousins with one of the landscapers, he would have to excuse himself from the vote. If someone was judging a play competition and had written a testimonial on how much they loved one of the competitors beforehand, they would excuse themselves from the judging process.

To argue against any claim of objectivity further demeans the process.

First, it's not objectivity that's required; it's being able to judge fairly. The reason we strive to prevent nepotism is because we realize that people cannot, by and large, decide justly because "blood is thicker than water." That analogy does not fit. If objectivity were what was desired, they'd bring in people with no knowledge of gaming whatsoever. What the process needs is fair expertise, which connotes a knowledge base informed by judgments on the worth of past material.

Second, the judges are elected and run on specific platforms. My understanding, having never voted in these elections, is that all candidates say that they like certain kinds of products and specific products. It's actually a job requirement. The Animalcast statement is a nonissue.

Third, the printed testimonial is the more serious charge. But there's no financial considerations flowing to the judge from the product. And the fact that the judges do not make the final decision--the voters do--makes its impact much less. They propose, the voters dispose, as the saying goes. Moreover, the single judge has no ability to unilaterally push that one product before the voters, which further undermines its impact on the process. If there were a smoking gun here or evidence of the product's unworthiness, that would be one thing. But neither have been shown.

So, thus far, your statement that "The ENnies are crap" is unproven.

 

[Meghan]

The statement "The ENnies are crap" was made on a different board. The discussion about the products were done elsewhere- in a podcast and talked about on a board for that podcast- and are inappropriate in a discussion titled "Zachary Houghton resigns as an ENnies judge". If you'd like to debate the products, the information backing up the reviews, or any of that, by all means, come on over!

I believe Zachary believed what he was writing. We don't have the inner workings of the process or judges. If someone is claiming that things aren't nearly transparent enough and others come in and say "everything's fine. It's all fine here. How are you?" I'm not going to take that at face value either.

My opinions are just one small voice. A judge though resigning because he became disillusioned with the system screams to me the system needs improvement. Bend over and do what Zachary says? Hell no. But in the time this argument has persisted, actual changes for improvement could've happened. Even in a volunteer endeavor (I know that too well) if there is feedback staring you in the face after a few serious errors and you ignore it, then you have to anticipate the same mistakes and the same issues to come up time after time.

 

[justanobody]

Qualidar: Yes.

The free products the judges get are the entries that the publishers submit. Review copies, in other words, provided so that no judge's ability to participate in the selection has to depend on his or her disposable income.

Now, I can understand that it's easy enough to check out a link online, but the quality of the site or podcast viewing/listening experience could be affected by the quality and speed of the judge's internet connection. So I don't exactly think that asking the submission to be on a cd is off base. The podcasters really should want the judges to be able to review at their own best convenience and in the best conditions they can manage.

As far as thinking of it as requiring every entrant to have put up something of value, whether in the form of the product in question or an entry fee, wherever that fee goes, I don't see any problem with broaching the discussion. Requiring some kind of value from everyone would fit some definitions of fairness, so I don't see it being a discussion to avoid, even if the idea is eventually rejected.

Personally, I'm not sure that resigning and letting a blog post be the primary notification mechanism for it is really a good idea or tactful plan.

True judge connections are a bit of a problem in a sense, but it also can help show the quality of an online product. If it is so large to not accommodate the worst of connections them it may mean an online offering needs some work for whatever reason. Take ENWorld for example. Didn't we recently have a lite version being run to compensate for hardware failure?

While it wasn't the intended design of the site, it was something to make sure people had access to the site so while a judge for physical products has everything in hand and needs nothing else, I was just thinking those added disadvantages that online media has that would or maybe should be a part of the review process. While a CD of the material can show the final product, unlike getting a book at a local store, the online media has a different dilevery method that must meet other standards than a physical product so that it can get to people who are interested in it.

It would be a sad case for someone with a podcast and only 2 gigs bandwidth per month to have people flock to it and the judge not be able to see it, but would show something about the podcast. It is a sad thing to say, but that podcast may need some work for it to be accessible to people other than a judge who could get it on a CD.

The awards should be based in part on performance correct?

A book or such that would be presented in newspaper cut-out leters will probably not go over well as a whole product because of how hard it is to read, and likewise a podcast that does not have enough bandwidth to support it probably isn't taking the proper steps to deliver the digital fan product....

I also understand that judges need something to look at in the form of a physical product, but didn't figure they got to keep them, and maybe all those products were auctioned off to support the awards or something else. Like a baking contest the judge has to sample the cake/pie, but doesn't get one to themselves, but there is one for each judge to sample, or Golden Demon each judge does not get a mini or diorama, but must share viewing of the entry.

I just think the delivery method of digital media is an important part of the quality of it.

Looking at Dungeons and Dragons Insider you can se a lot about the quality of the service that people are talking about comes in the form of how it is delivered to them including the latest feedback requesting article.

Judges have to have something to judge after all.

For the fee to enter one type of entry and others not have an out of pocket fee seems kind of funny, but again it is the different forms of media. Assume someone were to ask a podcaster to mail in their web server that houses the podcasts to compare to a book from say Green Ronin.

You really can't equate physical media to digital media in every way because of the differences in how they are presented to the world.

Podcasts aren't that much or PDFs to send in a copy on CD if needed, but what of streaming media that does not allow for download because they don't want their "cast" replicated?

So I am just saying that another option exists for digital media that does not for physical product.

Of course and digital book in eBook/PDF format could be printed and mailed in, and likewise any physical book could be digitized and placed on a CD so there is overlap for some, but not all in comparison.

Also digital media outside of the local area (country/state/etc) would be easier to get in than physical products.

Say someone in a country couldn't mail to the judges for the Ennies? Or what they mail takes so long through customs or has duties owed on the customs? Does the Ennies turn those entrants away because they don't want to pay the charges to get the mail? Where a submitted link costs nothing between countries to get it to anyone.

 

[Arnwyn]

Aside:

And, frankly, I'm not going to listen through a 78 minute episode to find the part where you provide the evidence that you should be citing in one of your lengthy posts on the subject in two forums. If you want to persuade audiences of the rightness of your opinions, you need to make it a bit easier for them to get at the meat of your argument than that. What. Is. The. Evidence. You. Cite. There?

Oh, give me a break. I was with you all the way, until here. A podcast is a podcast. Listen, and you'll likely get your answers. If you don't want to listen, that's your problem. Expecting anything more is not only unreasonable, it's flat-out silly.

Love this thread - it's always fun reading about a good "scandal" - real or imagined!