Opening can o' worms

[Dextra]

Evaluation Criteria

When I asked about suggested evaluation criteria, I was hoping the EN Worlders would post what they would like to see used. If the judges are there to represent the people, the people should speak up and say how they want products judged. Of course, each judge has their own preferences and values, but this could at least serve as a starting point, especially for newer judges.

I'll start you off with some things I as a voter (YMMV, please discuss) want addressed in a few sample categories:

-General philosophy: should price/value for money enter into the evaluation?

-Cover: not just the art, but the text, font selection, etc. Does it convey the feel of the book? Make me want to pick up the book? What about the back cover?

-Production Values: recycled paper? biodegradable ink? glossy paper, hard cover, solid editing, creative/effective layout, pleasant graphic design, binding doesn't drop pages, etc.

-Writing: descriptive prose- evocative, sets the mood, supplements the rules nicely, explains concepts well, fits the style of the book, etc.

-Adventure: easily used out of the box with minimal-to-no tweaking as DM. Minimal-to-no rail-roading. Feasible plot-hooks. Fun/playability. Appropriately indicated challenge ratings (where applicable).

-Publisher: supporter of online gaming sites (not just ENW), maintains up to date web site, good communication with fans, regular release schedule, excellent products, supporting organized play & game days, charitable work, treatment of freelancers, etc.

 

[trancejeremy]

I'd think we very much don't want to pick random judges. We want to pick judges with good judgement. And, as I think was first pointed out by Teflon Billy, one generally becomes popular around here in large part because your views on gaming are representative of a goodly segment of the community.

Well, I disagree. I think we get the judges we do simply because ENWorld is a very clique-ish place, and all the judges have been members of the "in-clique" so to speak.

They also get a huge boost from their being affiliated with ENWorld somehow - either mods or staff reviewers.

 

[Dextra]

Well, I disagree. I think we get the judges we do simply because ENWorld is a very clique-ish place, and all the judges have been members of the "in-clique" so to speak.

They also get a huge boost from their being affiliated with ENWorld somehow - either mods or staff reviewers.

I'm fairly certain the voting will be quite different this time along, what with us allowing anyone to submit themselves for consideration (provided they get a second), and anyone being able to vote for them (IP tracking will be in effect again). I doubt that being an EN World staffer will hold much clout, say, with the voting block from The Forge. It'll be a very interesting election!

 

[The_Universe]

Judge Selection: Should judges be able to self-nominate, or should they receive a nomination and second before being put on the ballot?

Self nomination is OK by me - the vote process should sufficiently weed out judges who don't have the reputation/skill/whatever to serve. In short, I don't see any particular benefit to adding an additional hurdle to the judge selection process - I certainly can't/won't criticize the judging panel that has arisen from the current process.

d20: I would like to keep the best d20 category as a nod to the origins of the Awards as well as reflect the unique advantages and disadvantages of using the d20 STL, but redefine it so that a product cannot compete in the Best Game as well as Best d20 Product. Please, no conjecture about 4th ed and how it'll affect future Awards, let's focus only on the 2007 Awards.

I have to admit that I'm not a big fan of this proposal - to me, the two categories are measuring different things, using entirely different metrics. In the Best D20 Category, we're looking at how a game uses the strictures of the D20 system, whereas in the larger "best game" category, we're measuring an overall experience. To deny D20 games the chance to compete in the "Best Game" category at least implies that no D20 game can be the best - that's just silly, and a slap in the face to D20 publishers.

I agree that the "Best D20" category should be kept as a nod to both a) the structure of the market, and b) the origins of the awards. However, if D20 games are to be barred from competition in what is now called the "Best Game" category, it should at least be clarified - make the categories "Best D20" and "Best Non-D20" so that the exclusivity of the categories are clear.

Mega Books: With the increasing number of mega-books with content spanning multiple genres, should we limit the number of categories an individual product in which a product can compete in the "Genre" domain (best adventure, best campaign/campaign setting supplement, best monster/adversary, best supplement)? Ie. publisher picks one? Or do we reward the products for their content, so in theory a Ptolusesque product could receive nominations in all?

I say you continue the current track, and award for content, even if it is a genre-spanning product. As a consumer/player/DM, I want to know what these products really deserved to win for - I'm not interested in "spreading the love;" instead, I want to know what is genuinely the best product out there, even if it is a multi-purpose product.

Additional Categories: I am contemplating adding the "Best Paraphernalia" category to the list to include items such as T-shirts, RPG fiction (and comics), dice bags, RPG movies, etc. In other words, the Aid/Accessory category would be open to products that enhance game play, paraphernalia to products that enhance gamer lives.

That sounds amusing, at the very least. I say do it - though I'll bet somebody 10 bucks that something from Order of the Stick wins it if this category is created.

Category Definitions: how are we doing with the definitions as is?

I like them. They seem perfectly reasonable to me, and they certainly make sense as Kevin explains them from the podium. I wouldn't change a thing.

Category Evaluation Criteria: by what criteria do you think each category should be evaluated?

The existing criteria seem to be working fairly well. I see that there's some dissatisfaction with Shackled City, for example, counting as a campaign setting. However, to me, this is exactly the kind of genre-spanner that, if it is as high-quality as it seems the voters thought it was, deserves as much recognition as the community chooses to heap upon it. After having read the thing, I'd have to agree with this year's Ennies judges, and count it as a Campaign Setting, as well as an adventure. Of course, that's just an example, and a long way of saying I think the present definitions are fine.

 

[The_Universe]

Well, I disagree. I think we get the judges we do simply because ENWorld is a very clique-ish place, and all the judges have been members of the "in-clique" so to speak.

They also get a huge boost from their being affiliated with ENWorld somehow - either mods or staff reviewers.

I think the accusations of "cliquishness" are somewhat misplaced here. I certainly don't think we ought to penalize a potential judge for being involved in the online community that has been instrumental in the development of these awards, which seems to be the alternative you're suggesting.

I think Dextra's attempts to open up the voting and judge nominations to a wider pool of people are the best we can do to allay your concerns - but I bet the votes will turn out largely the same. Not because of any secret "in-clique" here at Enworld, but rather because our judges have a genuinely good reputation.

 

[Mark]

One or two "new blood" positions on the nominations committee would be a good idea; that is, one or two positions guarenteed to be filled by people who have never been on the nominations committee in previous years. I think a lot of people who might not normally throw their hat into the ring would do so if there was an assurance that there would always be some new blood on the committee.

Staff reviewers on the committee? I've never been comfortable with that idea. We hear from them all year long. We know they all have some companies they don't care for and some who won't send them products. It's just always felt like there was an inherent conflict, though I am sure it is unintentional and is not meant as an idictment of any particular individual or company. Other staff members like Moderators, Admins, Newshounds, etc.? That might or might not be a conflict depending on which position they hold and what interaction they have with publishers while wearing their staff hat.

Now, I know some will say that a lot of the people that have those possible conflicts of interest are some of the most highly qualified people for the committee, and that's true in some respects, but there are reasons why people who work in other aspects of the industry aren't eligible, too. Safer to avoid the appearance of potential conflict and just disallow the possibility of being on the nominations committee to anyone who works for the site, as much of a disappointment as that might be. Let's face it, with a (supposed) membership of around 40,000 members and it being more vigorously opened to potential committee members from other sites I don't think it will seriously deplete the pool of talent available to sit on the committee. It's all about avoiding the appearance of a conflict of interest, despite anyone's best intentions, and knowing that with (probably) 50,000 + other people to choose from we'll do just fine.


Obviously, the opinions in this post are such that those they effect (staff members, previous committe members like myself, etc.) will feel compelled to object and I can only say that I hope you understand the positions and can objectively accept them as valid opinions. I'll post on some other ideas as I recall them from previous posts or new ones if they pop up. Thanks for considering all of these suggestions, Dextra, and I'm sure you'll come up with a good, unbiased policy that has the long term growth and interest of the ENnies at heart.

 

[Dextra]

Ah, but the way I see it, a d20 product shouldn't be awarded Best Game, because it isn't a game. A d20 product -using the strict definition of d20, ie. having the logo on it, not having character creation or levelling rules, etc- is not a complete game in itsself. Nor should its designers be able to take credit for it being a game, because chances are, they didn't create d20- WotC did.

d20 products can still compete in Best Product, as can Best Game, but Best d20 and Best Game should be mutually exclusive.

I have to admit that I'm not a big fan of this proposal - to me, the two categories are measuring different things, using entirely different metrics. In the Best D20 Category, we're looking at how a game uses the strictures of the D20 system, whereas in the larger "best game" category, we're measuring an overall experience. To deny D20 games the chance to compete in the "Best Game" category at least implies that no D20 game can be the best - that's just silly, and a slap in the face to D20 publishers.

I agree that the "Best D20" category should be kept as a nod to both a) the structure of the market, and b) the origins of the awards. However, if D20 games are to be barred from competition in what is now called the "Best Game" category, it should at least be clarified - make the categories "Best D20" and "Best Non-D20" so that the exclusivity of the categories are clear.

 

[bowbe]

Rather than Best Game...

How about Best New Game

This being a new spun game and not a revision of some other rules set or classic. I'm flexible on this however because hey, I'm a fan of Deadlands. Their new version wouldn't by definition be a "new game" but would get a vote from me in some way shape or form were it nominated.

While I'm thinking of it, why not have:

Best Game Derived from a Previous Source (should one come up)

This year we had 2nd editions of M&M and Spycraft... both old games but newer and better versions of those games. Likewise the well deserved gold for Shadowrun. None of those were "new" games but all are good and all are popular and all deserved some form of nomination in my book.

So

Best New Game

Best D20 Product

Best Derived or Revised Game (Thinking Runequest, Deadlands here...)

Best OGL Game ( I know ruffles were feathered or vice versa by some folks who flatly do NOT want to be associated with the d20 license but are in fact games derived from the OGL). They deserve their seperate place.


I realize it's 4 awards but I think it would deliniate what they were nominated for and make a good number of folks quite happy.

Also...

No to the T-shirts, coffee mugs and beer coozies or chip dishes.

Yes to differentiating an award for miniatures and one for game organizational stuff (Best Aid or Accessory).

Miniatures being a Best Miniature award.

Mini's really need their own award. For everyone who claims they never use miniatures or print out Claudio's awesome counters to run their game because it "cheapens" their imaginative RP experience I will show you somebody with a used booklet of graph paper filled with little arrows, circles, squares and the like they used to explain a complicated RP scene. Or a bag full of fuzz covered jellybeans that have served the same purpose as a miniature without the artistic endeavor taken to sculpt and create a tiny, beautiful (or even horrific) work of art.

Best Aid or Accessory should totally be the cool Game organizational stuff. All the trinkets such as spell cards, dunjinni, campaign cartographer and so on that make GMing and playing a game fun and efficient.

And you are correct Best anything could still compete for best product. No doubt about it.

Thanks for everything Dextra.

Peace

Casey

 

[Crothian]

Staff reviewers on the committee? I've never been comfortable with that idea. We hear from them all year long. We know they all have some companies they don't care for and some who won't send them products. It's just always felt like there was an inherent conflict, though I am sure it is unintentional and is not meant as an idictment of any particular individual or company. .

This might be a fast way to see a few Staff Reviewers retire. I know if this became a policy I'd step down as a Staff Reviewer in a heartbeat. I would go back to being a fan reviewer and frankly, other then the title I doubt much would change.

 

[Mark]

This might be a fast way to see a few Staff Reviewers retire. I know if this became a policy I'd step down as a Staff Reviewer in a heartbeat. I would go back to being a fan reviewer and frankly, other then the title I doubt much would change.

Don't quit too fast. Obviously such a policy would need some sort of clause where someone who had been in a position where a conflict of interest was possible would need to have an ineligibility for some period of time. At the least, since the period in which products are eligible has been going on for 4 months (?) already, it would necessitate disqualifying someone who had been a staff reviewer for part of the product eligibility year. It'd probably be good to mix it up a bit on the committee, anyway. I'm sure even members of the committee can agree that the same old folks out of 50,000 + isn't necessarily a good thing, as good as each member has obviously been in the past.