ENnies V - and beyond...

Psion said:
And yes, that gives it the fundamental quality of a popularity contest. I hold that you can't factor that out and have it be a "people's choice" award.

Oh, a popularity contest is fine. I think, though that we don't want it to be a marketing contest. Would we not prefer to be looking at popularity that is based upon the product's quality rather than the advertising budget of the company?

This is one reason why I respect WotC for staying out this year. They have such a massive edge in marketing that it might seriously skew the results. Merely the question of that arising could tarnish the reputation of the awards. Their staying out keeps our awards cleaner.

I would be interested in hearing about them, because you have me at a loss.

It seems to me that the Hugo Award voting procedure fits the bill.
------------
In summary -
You have some candidates in a cateegory. Let us say there are 5.

You may rank them in order of priority. You give priority 1 to the product you think is most deserving, priority 2 to the product next most deserving, and so on. I you are unfamiliar with a product, you don't have to vote for it at all. Just rank the ones you know.

You tally the votes for first preferences. If one product gets more than 50% of the votes, it wins.

If no product gets more than 50%, you go to the product that had the least first preference votes. You sort these by second preference, and those second preference votes are added to the previous first preference totals. If the ballot doesn't list a second preference, it is discarded.

Now, if one of the products has more than 50% of the remaining ballots, it wins.

Otherwise, we repeat the process, eliminating the least popular, and redistributing votes. If a ballot's 1st and 2nd preference have both been eliminated, you go to the 3rd preference, and so on.

Eventually, when you get to only two candidates, oen of them wins. Unless there's a dead heat, in which you give both products the award.

That all sounds complicated. And if you were dealing with the votes by hand, it'd be a pain. But electronically, it can be automated. The Hugos also have a "No Award" candidate in each category, if you feel none of them are winners, or feel that the category shouldn't exist at all.
------------

The upshot of all this is that it is impossible to vote against a candidate, as our current system effectively allows. The worst you can do is vote for them less strongly.
 

log in or register to remove this ad

Psion said:
I would be interested in hearing about them, because you have me at a loss.

There is no fair system I am aware of that will give a totally obscure product a completely equal playing field. The problem with the current system is that it has the potential to not only level the playing field for obscure products, but to give them an advantage. The fewer people vote for your product, the better off you are because it gives the votes of your "partisans" (the people who give it a 10) more weight. The more votes you get for your product, the worse off you are because it gives the votes of your "partisans" less weight and increases the chances that outliers will be trying to pull down your score.

There may be a weighting scheme similar to what IMDB uses that will obviate these problems. I don't know how it would work. I tend to think that assigning points is fairer, but it would make it harder on more obscure products.
 

AaronLoeb said:
The problem with the current system is that it has the potential to not only level the playing field for obscure products, but to give them an advantage. The fewer people vote for your product, the better off you are because it gives the votes of your "partisans" (the people who give it a 10) more weight. The more votes you get for your product, the worse off you are because it gives the votes of your "partisans" less weight and increases the chances that outliers will be trying to pull down your score.

We could try ignoring the top 5% and bottom 5% of the votes for any given product.
 

Actually I thought we had this argument about the voting system last year. I agree with Aaron that it's flawed. Either lower the rating to 5 or make it so that the minority influence isn't so great. It's easy to sabotage other products by voting a 1 for them, and what we end up having are various voters voting 10 for the products they want to win and giving a 1 to other products just so to push down the rating and give more opps for the product they want to win to get the top position. Sure there might not be many folks doing this, but it's possible.

Tossing out the top and bottom 5% is also going to cause more complications.
 

Krug said:
Actually I thought we had this argument about the voting system last year. I agree with Aaron that it's flawed. Either lower the rating to 5 or make it so that the minority influence isn't so great. It's easy to sabotage other products by voting a 1 for them, and what we end up having are various voters voting 10 for the products they want to win and giving a 1 to other products just so to push down the rating and give more opps for the product they want to win to get the top position. Sure there might not be many folks doing this, but it's possible.

Tossing out the top and bottom 5% is also going to cause more complications.

Ya, we did have this coverasation last year and the year before.....

With the voting the obvious one lika ten for one and a one for the rest are automatically tossed out to protect the system some. Its not perfect, and changing it to a rating of 1-5 sounds like a good idea.
 

For some reason I can't post under my regular screen name so I'm posting my thoughts under this one.

There have been many well thought out ideas and proposals on this thread. Something I certainly agree upon is the need for a mission statement for the ENnies. The awards need a foundation upon which everything else is structured and a mission statement will help achieve that. Having Dextra assume management of the awards is also a very positive step and would serve to help get more publishers involved because of the way she runs her business.

I also think the categories need to be rigidly defined and clarified. This allows publishers to determine exactly which of their products is best suited for each category and it lets voters understand exactly what each category means. For example, when the category of Best Setting Supplement was announced, I took it to mean that supplements specifically designed for established campaign settings such as Forgotten Realms, Oathbound, Scarred Lands, Warcraft, etc. would be found here. I was surprised when the nominees included Corwyl and Crooks! (both excellent books) only because I hadn't considered them to be campaign setting supplements, rather I felt they were campaign resources designed to enrich the game overall. (*Note, I am not saying these products did not deserve to be nominated and it should not be taken that way. I am only using this as an example.) Had a definition of qualification been posted for each category, it would have made it easier to understand what did or did not fall under that category and even why. A rigid definition of categories will also serve to help the judges in determining exactly where a product fits. Using my earlier example, defining Best Setting Supplement as any product that is specifically designed to be used in an established and named campaign world would qualify Crooks! but not Corwyl and Thieves Quarter. However, if a category of Best d20 Resource were created and defined as any product that is generic in nature and may be used in any established name campaign setting without substantial changes, those books would qualify as would Redhurst and possibly Omlevex (rather than Best Campaign Setting). Had definitions of the categories been established, there may have been more entries. For example, had a category for Campaign Resource existed, Bastion would have likely entered three additional products that were not included with our present entries.

I think there should also be a concerted effort to get more publishers to participate in the awards. The ENnies are still young and as such, are still struggling for acceptance among the publishers. While great strides have been made, I also think that unless publishers such as Wizards of the Coast get involved, there will still be the lingering spectre of legitimacy hanging about. After all, if some of the big names don't want to play, then how can anyone truly say that product X was the best of category Y? We need the ability to include everyone so there is no dispute as to who is the current king of the hill for that category.

As for requiring a submission fee on top of sending product, simply put, that won't fly. Why would a publisher send 6 or more copies of each product for consideration (at a cost of at least $10 per book to the publisher) and turn around and pay more for the privilege of being considered for an award? The more requirements and work that publisher must do, the less likely he will be to want to be involved. The ENnies are great, but I just don't see publishers lining up to subsidize them by paying a fee for each book considered. There has to be a different way.

I also agree that the voting is far from perfect, but there really isn't a full-proof solution to this issue. As long as voters must make choices, even if it's to write in their vote, there exists the possibility of charges of unfairness or favoritism. Perhaps a key to preventing this is by publicly showing exactly how voting is determined and scored. Transparency is always a good thing.

Lastly, an unpopular point that has been brought up in the past but still remains an issue to publishers is the presence of a panel of judges to determine the nominees. Certain companies feel they will never get a fair consideration as long as judges with clear biases against those companies are involved in the selections. Therefore, they will not enter the awards. This goes along with some of the criticisms that Monte Cook brought to light on his site's forums. Why should a publisher feel obligated to enter products for the ENnies, when he is publicly criticized on a regular basis by the EN community or on EN reviews? Again, the ENnies need to work towards getting everyone involved and not just their favorite sons and daughters, so to speak.

While I readily admit that the judges do a remarkable job in the short time they have to work with, it is a thankless and very demanding job. Objectivity is a must. Judges must be willing to set aside all preconceptions and biases and be willing to read the entire product regardless whether they initially like it or not. If you read the first couple of chapters, decide you don't like it and move on to the next submission, who's to say that you aren't missing out on something great later in the book? An example of this is Guardians of Order's Slayers d20. The first part of the book is geared towards the anime series. It's only when you get to the latter half of the book do you discover the magic rules and the rules for psychological warfare, which are some of the book's greatest strengths. If a judge were reading this book and decided he didn't like the episode summary in the beginning and put the book down without finishing it, its chances for nomination just went down the drain, so to speak. The reason I bring this up is because such a comment was made by a judge about not finishing a book he didn't like. There must be total commitment or again, the legitimacy of the awards fall into question.

Overall, I feel the ENnies have made great improvements and hope they will continue to do just that. Great strides were made this year and I am confident that greater strides lie on the road ahead with the proper direction and management. I look forward to seeing what comes of these discussions. :)
 

I am glad that this discussion is occurring, and am watching it quite intently. Thank you everyone who has expressed their faith in me.

Honestly, I have no idea if I'd be acceptable to manage the ENnies. I certainly wouldn't want to have any part in the nomination process nor the judging, so hopefully my being a freelancer and working for ENPublishing wouldn't affect my eligibility. I just want to make next year's process easier on the staff, and the ceremony the best ever, and help nurture the ENnies so they can grow to their full potential. If a bunch of publishers spoke up and said that it would be inappropriate for me to participate, or that they hated my guts and wouldn't want to support an organization with me at the helm, then I would certainly bow out.

Also, please be advised that my comments and suggestions were merely that. There are a lot of people out there much smarter or more creative than I am with some brilliant ideas and more experience. I just threw out some ideas (hey, it took us fifteen hours to get home, and I drove most of it, so had lots of time to think!) in the hopes of prompting some discussion- and I guess it worked! Some of ideas I believe very strongly in, like the need to improve our image and visibility. Others, like the categories, fees, nomination process: totally negotiable.

I don't know when Russ'll be making his decision as to the direction the ENnies will be travelling next year, and who'll be the navigator. This is pretty early in the game. Personally, I think the sooner the better so we can have a year to collate feedback, raise some money (I've had some more ideas, but won't show my hand yet) and raise the ENnies' visibility (even more ideas). I suspect he's watching these discussion and thinking hard right about now.
 

There are many topics here to which I could lend an opinion. So many my head is swimming.

I think i'll chose one to address right now though. Wizards of the Coast.

During Gencon, I casually asked someone at WOTC why they submitted no products this year. I was told that the general feeling was that WOTC considered the Ennies an award for small or independent publishers (dont quote me, i belive i'm paraphrasing from memory).

Issues with WOTC running for ENnies have been brought up before, and they are sound.

If WOTC enters, and sweeps, whats the point for everyone else?
If WOTC enters, and doesnt win everything, it will appear as if the awards are "rigged" against WOTC. Why? Because WOTC is "considered" the best by virtue of the fact that D&D is theirs.
If WOTC doesnt participate, the ENnies appear to be unsupported and illigitiment.

Additionally, I dont think its been noted, or addressed, if the ENnies have any actual financial impact on a product, via increased sales or increased shelf life.

I do belive that this issue needs to be solved within the next 6 months.

One possible solution is creating an awards catagory between large and small publishers.
This "could" work, but Large and Small publisher really has to be defined. The original qualifier doesnt really fly, especially with POD books.
To answer that question, we would really have to wonder what seperates WOTC from Malhavoc and Green Ronin and Bastion Press among others. And what seperates these companies from Expedious Retreat Press, EN Publishing, RPGObjects, etc. For example, where would RPGO fit? Where would Bastion?
Doing something like this may alienate some companies. Who knows. IN the end, the line really stands at how much money a company has to devote to development and printing. Their staff, etc etc.

Another option, would be to get WOTC involved with the awards without participating directly.
Either by allowing their products to be entered by the authors themselves, or by asking WOTC to help Fund the ENnies. If they wont submit products, then funding the ENnies would certainly be a way to openly show support for the awards.
 

Krug said:
Either lower the rating to 5 or make it so that the minority influence isn't so great.

Tossing out the top and bottom 5% is also going to cause more complications.

The current IMDB system makes sense for IMDB because its goal is to come up with an average public "grade." That's not a goal of the ENnie voting, I don't think, as the final scores aren't released. As such, I don't see the benefit in a "grade" system. Even if you make the grade 1 through 5, it is still possible to skew the results with negative votes.

In general, I think when creating a voting system for awards, there should not be a way to vote against something. A "grading" system will always involve a "bad grade." Instant run-off voting, as Umbran suggested above, is a good system that keeps the power of the vote in the hands of the majority.

Dropping the top and bottom scores might work, though it disenfranchises voters who really do think something is worth a 10 or is the worst product ever created. Better, I think, to use a voting system less prone to manipulation through outlying votes so that everyone's vote is counted and counted equally.
 

BrooklynKnight said:
If WOTC doesnt participate, the ENnies appear to be unsupported and illegitimate.

I don't think this would be that much of a problem, if they simply sat it out. They're right - compared to WotC's sales figures, EVERYBODY is "small press" (including Steve Jackson and White Wolf, as big as they are).

To answer that question, we would really have to wonder what seperates WOTC from Malhavoc and Green Ronin and Bastion Press among others....IN the end, the line really stands at how much money a company has to devote to development and printing. Their staff, etc etc.
Actually, I would instead say that it's not their marketing budget - it's that very network externalities that makes d20 STL work for them that also puts up a wall between them and any other d20 system/OGL publisher.

Either by allowing their products to be entered by the authors themselves, or by asking WOTC to help Fund the ENnies. If they wont submit products, then funding the ENnies would certainly be a way to openly show support for the awards.

I would say the submission of products until individual names is not a good solution, specifically for the reasons Ari (Mouseferatu) and myself gave - the whole "credit where credit is due" issue. And it would be REALLY weird to list everybody who worked on a product - but NOT mention who made it. :confused:

If they volunteered to kick in a small contributory funding for it ($300-$500 a year?) in reciprocation for them being mentioned as a sponsor in addition to Gencon, or some such, that would be excellent - but if they don't, I don't think it will affect the status of the awards. There are a heck of a lot of artists, editors, and designers out there working hard, and to not be overshadowed by WotC is a more admirable goal than striving for WotC's inclusion.

I don't feel badly for people at WotC like (freelancer) Keith, or James W., or Darrin D, or Bill Slavicsek; I feel for the artists and editors who work for WotC who are probably more unsung than the artists and editors for most d20 publishers! :)
 

Remove ads

Top