[ENnies] WotC on the outs

[ColonelHardisson]

Despite Wulf's disagreeing with me, there were many publishers (prominent ones) at the awards that I spoke with that felt they never had a chance to win after the first few announcements came through. Considering the cost incurred to the publisher to submit a product for consideration (number of copies sent to the judges, shipping, etc.), there are several who will not do it next year unless changes are made.

I need to address this in another way. If these publishers do not speak up and let Morrus know what they took issue with, how can any changes be made that address any of their concerns? If they have something to say, it would be most helpful, and most constructive, to simply post here on this thread. I encourage you to let them know about this thread if they are unaware of it.

 

[Psion]

Just playing Devil's Advocate for a moment...

If the ENnies are to truly be a fan-based award, then why have judges at all? Do the ENnies awards require X number of judges to tell everyone else that these products are worthy of an award?

Because the judges will have seen all of the products. The same is not true of a straight popular nomination.

And judging from some chat at the awards ceremonies this year, some industry luminaries like that a lot, in contrast to that other major RPG award...

You know, a lot of things this year were very last minute, and were in part based on arguments never fully resolved. All that considered, I think that the awards turned out rather well. I think we are at the stage were we need little tweaks, not huge changes.

I am sort of bummed that some people are tearing down the awards, when here I was thinking things turned out surprisingly well. But I guess that is to be expected.

 

[Ghostwind]

Okay, this is not an attack on the judges or the winners, but let's take a look at the whole process and how it is perceived from the Devil's Advocate role once again...

First, a panel of judges are selected to determine/select the products that are eligible for nomination. Exactly who is chosen as a judge is dependent upon the number of popular votes they obtain during a brief "election" by the EN World community with the only information of their qualifications coming from a self-explanation as to why they should be allowed to serve as a judge.

Second, those d20 companies who wish to participate in the awards must then send multiple copies of whatever product they wish to be considered to those judges. This represents considerable investment and expense (especially for an overseas company such as Mongoose). The judges then review this mountain of material and choose what products are suitable for nomination in particular categories in their professional opinion.

Third, once the nominations are announced publishers begin to rally their fan base for votes by encouraging them to vote for the products they have nominated (which includes registering for EN World if they aren't a member).

Fourth, voting begins with little notice and a small window of opportunity to vote. Additionally, once you've begun the voting process, you cannot stop or restart due to the security precautions put in place (which means those who are cut off from dial up interruptions are out of luck). Individuals are asked to rank the products with a numeric number indicating familiarity. (I also remember Morrus indicating that if a certain vote seemed too skewed, ala one 10 and the rest 1's, it would be subject to possible deletion.) Also, the only ones allowed to vote are registered members of the EN World community, representing only a small percentage of actual gamers who role-play.

Fifth, the awards winners and second place finishers are announced. It is assumed that these represent a straight conversion of the total amount of votes received based upon the limited window.

Now, to further stir things up and make folks critically think about this, where are the areas that are subjective in nature and potentially skewing any result? First, we have the judges who do their best to be objective but suffer from the same human frailities as the rest of us. The judges, like reviewers, are going to be partial to products that they are most familiar with. This panel determines what products end up in what categories. For example, the nominees for Best Graphic Layout consisted of Monsternomicon (Privateer Press), Freedom City (Green Ronin), Farscape (AEG), Nyambe (Atlas) and Monsters of the Endless Dark (Goodman Games). These products were the ones that the judges felt qualified to be in this category (and art is a very subjective thing). The EN World public has no say otherwise. Going back to an earlier post, if Decipher had created a d20 LoTR game and it was not included, there is no way to write-in a vote or voice an objection. The popularity race begins based upon the selection and opinion of the judging panel (who do try their best).

The next area of skewing involves those companies that choose not to send material/enter the awards. Does this mean that they are less deserving of an award because they chose not to enter? Had Wulf not decided at the last minute to enter, HOHF: Elves would not have received a nomination. Furthermore, just because Wotc chose not to enter Silver Marches, for example, automatically disqualifies it from any award of excellence. Do you see the inherent issue here?

Thirdly, once the nominations are announced, the vote rallying begins. In a nutshell, the company that can effectively garner the most popular votes for their nominated products will win. It has nothing to do with excellence. As previously mentioned, Avalanche Press won an Origins award for Best Role-playing Supplement for a book that the ENnies would likely never even considered because it has a very strong fanbase among boardgamers that always vote in numbers. Does this mean that the current winners don't deserve to win? By all means, no. They put out a quality product. However, there is no way to determine whether those votes were because community members felt the product was THAT good or because they were supporting their favorite company.

Psion points out that the judges are the only ones to have seen all of the products submitted, whereas the gaming public has not. What effect would it really have if there were no judges? The casual gamer would then be forced to choose a d20 product for each category based upon his impressions of quality and not someone else's. Not having a judge panel means companies don't have to choose what products to submit for nomination consideration. No products submitted means that everything is open for consideration by the casual gamer based upon any pre-defined criteria (such as a product containing a copy of the OGL within its pages). Is it absolutely necessary to have judges select products or is their an underlying and unstated assumption that gamers aren't smart enough to accurately pick suitable products for nomination and subsequent wins?

Again, just trying to provoke critical thinking and discussion, not slamming the winners or judges...

 

[ColonelHardisson]

First, a panel of judges are selected to determine/select the products that are eligible for nomination. Exactly who is chosen as a judge is dependent upon the number of popular votes they obtain during a brief "election" by the EN World community with the only information of their qualifications coming from a self-explanation as to why they should be allowed to serve as a judge.

The fact that they are recognized by the community in general as being part of that community is an important distinction you're leaving out. The ENnies are, after all, not just the GenCon, awards, but the EN World awards. EN World is a community.

Second, those d20 companies who wish to participate in the awards must then send multiple copies of whatever product they wish to be considered to those judges. This represents considerable investment and expense (especially for an overseas company such as Mongoose).

Overseas companies like Mongoose have their distributors in the States send the copies to one of the judges (me, in this case). The expense isn't as great as you'd think - except for Morrus. He shoulders the shipping to all the other judges, which amounted to almost $1300 US this year.

The judges then review this mountain of material and choose what products are suitable for nomination in particular categories in their professional opinion.

Third, once the nominations are announced publishers begin to rally their fan base for votes by encouraging them to vote for the products they have nominated (which includes registering for EN World if they aren't a member).

Fourth, voting begins with little notice and a small window of opportunity to vote. Additionally, once you've begun the voting process, you cannot stop or restart due to the security precautions put in place (which means those who are cut off from dial up interruptions are out of luck). Individuals are asked to rank the products with a numeric number indicating familiarity. (I also remember Morrus indicating that if a certain vote seemed too skewed, ala one 10 and the rest 1's, it would be subject to possible deletion.) Also, the only ones allowed to vote are registered members of the EN World community, representing only a small percentage of actual gamers who role-play.

Again, this is the EN World awards, after all. Much of this was intended to ensure that the EN World community had a large amount of say in the awards.

Fifth, the awards winners and second place finishers are announced. It is assumed that these represent a straight conversion of the total amount of votes received based upon the limited window.

Now, to further stir things up and make folks critically think about this, where are the areas that are subjective in nature and potentially skewing any result? First, we have the judges who do their best to be objective but suffer from the same human frailities as the rest of us. The judges, like reviewers, are going to be partial to products that they are most familiar with. This panel determines what products end up in what categories. For example, the nominees for Best Graphic Layout consisted of Monsternomicon (Privateer Press), Freedom City (Green Ronin), Farscape (AEG), Nyambe (Atlas) and Monsters of the Endless Dark (Goodman Games). These products were the ones that the judges felt qualified to be in this category (and art is a very subjective thing). The EN World public has no say otherwise. Going back to an earlier post, if Decipher had created a d20 LoTR game and it was not included, there is no way to write-in a vote or voice an objection. The popularity race begins based upon the selection and opinion of the judging panel (who do try their best).

Most d20 publishers are made aware of the awards, if we feel they are unaware of them already.

The next area of skewing involves those companies that choose not to send material/enter the awards. Does this mean that they are less deserving of an award because they chose not to enter?

In a sense, yeah. Most movie awards have to be purposely entered by the filmmakers, for example. You can't win if you don't enter. Most movie awards I'm aware of don't have write-ins.

Had Wulf not decided at the last minute to enter, HOHF: Elves would not have received a nomination. Furthermore, just because Wotc chose not to enter Silver Marches, for example, automatically disqualifies it from any award of excellence. Do you see the inherent issue here?

No, honestly, I don't. The awards are for products that are entered. EN World is the largest d20 site on the net. Most d20 publsihers are aware of it, and those who aren't we've tried to make aware of it. Just about anything d20 gets mentioned at EN World eventually. I know that the judges did a lot of "beating of the bushes" to ensure that as many d20 publishers as possible were made aware of the awards. That is, when we became aware of a company that had not entered, an e-mail of inquiry was sent to see if they had any interest in entering. Did we contact every single company out there that had a d20 product and presumably was unaware of the ENnies? I doubt it. But we tried. We discussed this very issue from time to time - how to make sure everyone who could enter, and might want to enter, know that the awards existed, and how to enter them.

Thirdly, once the nominations are announced, the vote rallying begins. In a nutshell, the company that can effectively garner the most popular votes for their nominated products will win. It has nothing to do with excellence. As previously mentioned, Avalanche Press won an Origins award for Best Role-playing Supplement for a book that the ENnies would likely never even considered because it has a very strong fanbase among boardgamers that always vote in numbers. Does this mean that the current winners don't deserve to win? By all means, no. They put out a quality product. However, there is no way to determine whether those votes were because community members felt the product was THAT good or because they were supporting their favorite company.

Avalanche, which had a comparatively good showing in last year's ENnies, and which at least two of the judges (myself and Teflon Billy) have shown support for consistently, was contacted about entering. They never replied.

Psion points out that the judges are the only ones to have seen all of the products submitted, whereas the gaming public has not. What effect would it really have if there were no judges? The casual gamer would then be forced to choose a d20 product for each category based upon his impressions of quality and not someone else's. Not having a judge panel means companies don't have to choose what products to submit for nomination consideration. No products submitted means that everything is open for consideration by the casual gamer based upon any pre-defined criteria (such as a product containing a copy of the OGL within its pages). Is it absolutely necessary to have judges select products or is their an underlying and unstated assumption that gamers aren't smart enough to accurately pick suitable products for nomination and subsequent wins?

Of course not. But what you present is a situation which would result in exactly what most of the d20 publishers don't want to have happen - their being overshadowed by WotC. WotC would, by simple virtue of market penetration, dominate the awards almost completely. This was discussed after last years awards very vigorously. d20 publishers have, in the past, indicated they would be unhappy with such a situation, and presumably would cease to support the awards because of it. Going to a straight popular vote is the surest way to alienate the most d20 publishers. It isn't about the general community not being smart enough to pick nominees; it's about the fact that they can't possibly have seen enough of the smaller publishers' work to even be aware of them - A Magical Medieval Society is a wonderful case in point. At least having a panel of judges ensures that some of these smaller publsihers at least get some deserved attention. The awards are, after all, about supporting the d20 industry more than they are about simply picking the most recognizable product.

 

[Darrin Drader]

I think there should be another 2 or 3 non-open categories, and the issue with the art being neither open or non-open is also valid.

With regards to Steve's points, (Steve of all people should know not to take any of my rebuttals personally), I think that his logic is somewhat faulty.

Point 1: A panel must be selected if the vote is to be formatted as it has been. The people who have returned to these boards time after time, year after year, have shown enough dedication to the hobby that they should be considered "qualified."

Point 2: I don't see how this is either here or there. Anyone wishing to enter any sort of award is expected to do this. Call it a marketing expense.

Point 3: Again, this is no different than any other sort of publicly decided award. Movies, comics, etc. all invite their fans to participate after the nominations have been made. It's how it is done.

Point 4: This, I think, is a valid critique. Exactly who do the Ennies represent? Should you have to be registered on the boards? Is visiting the news page everyday not enough participation in itself to qualify you for voting? If you do have to be a board member, are there any particular bias on the boards towards or against particular publishers? Unfortunately personal bias is something that you just can't escape. I do think non-board members should have been allowed to vote, and I think that if the samer IP address tried to vote more than once, the most recent vote is the one that should count. By doing it in that way, it allows people to change their vote or try again if their ISP crashed.

Companies that choose not to submit their material are out. I don't see this as an issue. The alternative is that the panel must get together and discuss the products that they have purchased and them make a decision on who to nomiate accordingly. They must somehow have the clarivoyance to know which contenders are the best from an entire year and then make an investment in that product purely for the purpose of evaluation. I think that a bit silly.

Is it absolutely necessary to have judges select products or is their an underlying and unstated assumption that gamers aren't smart enough to accurately pick suitable products for nomination and subsequent wins?

Again, I think this goes back to the last point I made, but adds the further complication that the nominations will be all over the place. What's more is that if the awards are a straight popularity contest, then why bother having a nominations and then announcing the winners at all? Why not just tell people to nominate the product/publisher that should win the category and then announce the winner based on the feedback? I think that this method would be a step backwards since it would remove any standard that does exist in the awards.

 

[Teflon Billy]

Okay, this is not an attack on the judges or the winners, but let's take a look at the whole process and how it is perceived from the Devil's Advocate role once again...

"Devils Advocacy" involves not necessarily agreeing with the points you put forward.

I don't see that in the post I quoted from.

 

[Crothian]

Okay, a few random thoughts to a few of the issues brought up:

We need judges. I buy a lot of d20 products, but even I was familiar with just a portion of what was sent. There were many products that were nominated and won like Mutants and Mastermind that I would not have been familiar with to accureately vote on. I'm sure even guys like Psion, who gets a decent amount of d20 material, say a few quality books he was unaware of.

I personally contacted a few of the more populiar companies that had not entered, to remind them of the ENnies. Some choose not to enter, and that is their choice.

I do think people had enough time to realise when the vote would happen. It's not like it wasn't mentioned when the submission deadline was and that voting would follow a few weeks later.

 

[Monte At Home]

What effect would it really have if there were no judges?

So there would be no submissions from publishers, obviously. The voting system would rely entirely on people's memories and awareness.

Smaller publishers would have an even harder time getting an award.

Older products would suffer in favor of those that came out right before the awards.

I really like the judges/nomination process. About the only thing I'd do different is require a turnover in the judge's panel--not because I'm unhappy with the judges, but because it just seems fair.

Are the awards without problems? Of course not. Are they better than any industry awards system I've been a part of in my 15 professional years? Yes.

I too, by the way, think that it'd be fine if WotC products could be a part of the art and graphics awards. I like, however, that the awards focus on d20/D&D. If I had my way, those awards would be opened up to WotC and other non-OGC D&D related companies, but not to the entire industry. The industry as a whole has awards already. It would be jarring, I think, to see HeroClix in the awards somehow (I use that as an example because it's something I've worked on).

 

[2WS-Steve]

First off, the ENnies have, in just three years, become one of the best awards in this industry. I feel like I can just use the nomination list as my shopping list with confidence that I'll end up with a set of books that I'll like (or, in many cases, a set of books that I already own and like).

1) I like the two-tier process. It does a nice job balancing the benefits of a popular award with the benefits of a critics award.

2) It seems like WotC should be able to enter more products. For example, the Epic Level Handbook and Deities and Demigods are due to be added to the SRD. Certainly once they get there, they're eligible but that creates this weird result that they'll be eligible next year even though they were released last year.

 

[kingpaul]

I've read through the thread, and here are my thoughts.

I do think that the window to vote was a bit small. I was on vacation where I didn't have 'net access for a week. I got back on the last possible day to vote. IIRC (and its more than likely my memory's faulty), there wasn't an advanced warning of when the voting was to commence and how long it was to take place.

I like the focus on the d20 aspect of the industry, but I think the 800 lb gorrilla should be eligible for more events. Let's face it, without WotC's OGL, the rest of us wouldn't be producing D&D-like stuff. They set the bar, and its continually getting raised.

I think the 2-tier system is good. I know I don't have the $ to buy every product for myself (although I wish I did, but my wife would really roll her eyes at me then ). This gives us a panel of people who are guaranteed to have seen all the products up for nomination.

As for the products up for nomination, I think the current method is good as well. If it were reversed, then the judges would need to go to the publishers and ask them if they wanted their product in the selection process. This, IMO, saves a step by have the publishers send the products in.