ENnie Nominations!

As others have noted, this system very effectively jettisons the "one person one vote" principle that is the foundation of every fair voting process. I *really* appreciate y'all trying to develop a fair and representative process, and I know it isn't easy. In my humble opinion, though, this was not the best choice. If it's a done deal for this year, that's fine -- but I hope it can be reexamined next year. Personally, I still don't understand why "the most popular stuff always wins!" is a legitimate criticism of the outcome of a popular vote.

So, are there any guidelines? As voters, are we expected to vote as objectively as possible, or are we expected to maximize the value of our vote? Different posts from the organizers and judges seem to indicate different answers to this question.

Greg
FFG

P.S. Thanks very much for the nominations. I know everyone involved really appreciates it!
 
Last edited:

log in or register to remove this ad

Morrus said:
Certainly a 10-0-0-0-0 vote is going to attract attention. I will be gobsmacked, in fact, if anyone manages to give any product on the list a 0. At most, 0's should be very occasional and won't have much effect.

I agree, but then why include a "0" as an option? All the nominees look really good (congrats again, guys!)

If you've already removed those that might deserve something in the 0-4 range, then why include it?
 

FFG Greg said:
Personally, I still don't understand why "the post popular stuff always wins!" is a legitimate criticism of the outcome of a popular vote.

I agree with you-- but it is easy to explain: By the nature of the distribution process, a lesser entry with wider distribution will receive more votes than a superior entry with a smaller distribution.

Despite the fact that I am in the "smaller distribution" crowd, I don't really have a problem with that. You put it very succinctly-- it's a popular vote, after all.

As I have said many times before, it is the nomination (by the panel of informed judges) that counts, not the outcome of the popular vote. I don't expect to win for either of our nominations, but I don't expect that to lessen my joy and amazement at even being nominated.

Wulf
 

Wulf Ratbane said:


I agree with you-- but it is easy to explain: By the nature of the distribution process, a lesser entry with wider distribution will receive more votes than a superior entry with a smaller distribution.

I hear you -- I was there last year when Wizards (not undeservedly) swept the awards. However, its a popular vote: when you say a "lesser" product with wider distribution might win over a "superior" entry, it seems to me your quietly introducing criteria and values from other awards processes. In a popular vote, you should expect that "best" will equate to "most popular."

That said, it's cool with me if the ENnies folks don't really want a popular award. I think EN World could easily (and has) put together a voting "academy" that would be legitimate and respected within the community. Let them choose the winners, just like the Oscars.

Greg
FFG
 

First off, wow! How cool to get nominated for Book of the Righteous. Thank you so much to the judges and to Morrus for throwing this very cool shin dig.

To Greg: Respectfully, it depends on what your definition of "fair" is when you say one person one vote is the foundation of all fair voting mechanisms. Weighted voting systems have a long and happy history and are certainly fair if you're trying to get a statistical sampling of the "middle" rather than the fringes and if you're trying to give voice to the minority as well as the majority.

Voting systems should be built to answer questions. More often, they force questions. For instance, last year's voting system (I think) wanted to ask the question "what is the best product in the opinion of the ENWorld community?" It ended up answering the question, "What product is the best in the eyes of the most members of the ENWorld community plus voters from outside this community who have been directed here by publisher web sites?"

A weighted scheme seeks to answer the question "What's the product that's rated highly across the board, not just by the simple majority." It's an effort to achieve consensus rather than simple majority, to find an answer nearly everyone feels good about. I think that's very fair.

The difficulty with the 10-0 ranking system in most contexts is that it is extremely gameable by the minority, yielding the same problems as one person, one vote only worse -- rather than the minority getting stuck with the choice they disagree with made by the majority, the majority can get stuck with a decision made by a passionate majority willing the game the system. The way to avoid this sort of thing in weighted voting is to have very small gradiations (1-3 for instance - the wider the margin available, the more gameable the system) or to require voters to rank the products sequentially so that all products receive votes.

BUT, I think what Morrus is counting on is that this year the voting is actually closed to ENWorld members (if I recall correctly). I think the assumption is that community members are going to be faithful and fair and not game the system. This is a fair assumption, I think. It will likely be a tiny fraction of people who will vote dishonestly (Let's be frank: 10-0-0-0-0 would be a dishonest vote looking at this year's nominees. Sure, I love Monsternomicon. Sure I'll rate it highly. But I would be genuinely insane if I thought its competitors deserve 0s. All of the categories include incredibly good products, as everyone has commented. Differences of opinion are of course the point of voting in the first place, but I defy anyone, no matter how passionate their opinion, to make a rational argument that any of these categories feature a genuine 10-0-0-0-0 disparity.). So, given that voting 10-0-0-0-0 is pretty clearly gaming the system, remember, you will have logged in: Morrus will know who you are. :)

But beyond that, the question we're trying to answer is "what is the product the ENWorld community thinks is the best of these nominees." Perhaps the feeling on Morrus' part is that if this community wants to game the system, or members of it do, well, that's a reflection of what the community wants to do. Personally, I'm HIGHLY prejudiced toward Green Ronin products (the company being owned by dear friends), but as a community supporter, I'm not going to go voting 10-0-0-0 for their products. I have faith that my fellow ENWorld members will be similarly fair.

Aaron
 

FFG Greg said:


That said, it's cool with me if the ENnies folks don't really want a popular award. I think EN World could easily (and has) put together a voting "academy" that would be legitimate and respected within the community. Let them choose the winners, just like the Oscars.

Greg
FFG

If you saw the discussions about the ENnies process after last year, and especially before this year's awards, you probably saw that the notion of a "judges choice" set of awards that were chosen concurrently with the popular vote had an overwhelmingly negative response. If the awards were entirely chosen by judges, with no popular vote, you can imagine how negative the reaction would be.

The methods Morrus has used to try to make everything fair haven't been about eliminating the influence of popularity, exactly, but rather the influence of familiarity. That is, companies that can afford big marketing campaigns will be familiar to voters, while small companies will never have been heard of. What happens is that people will vote for something they've heard of, sight unseen, over something they haven't heard of, sight unseen. The current method attempts to address that, to give the small companies a chance.

When it comes right down to it, though, the vote relies upon people voting their conscience. That is, the best case scenario is that the ENnies nominations will prompt people to actually check out the material before they vote, instead of simply voting according to what they've seen in an ad. Since the ENnies are intended to both honor and support the industry, getting people to explore the choices out there is the goal, more than simply honoring the most popular items, in my opinion. Of course, many of the most popular products are, indeed, very high quality. So, too, are there many high-quality products that you've never heard of that deserve a fighting chance. I can say that this is very true now - the quality of d20 products across the board has been raised greatly even since just last year. That quality seems pretty evenly distributed, even amongst products I'd never heard of before.

I hope any of that makes sense or is relevant.
 

Morrus, I would seriously recommend you reconsider the voting procedure. The method you suggest is inherently flawed, possibly even more so than the standard 1-vote majority voting.

Two excellent alternatives are (1) the "checkbox" method Psion thought was going to be used. It's called "Approval Voting" and is widely used for various scientific societies where people actually do put some thoughts into which voting method they use (including IIRC the American Statistical Association and the U.N.). (2) the strict ranking method suggested by jgbrowning and others. Especially if those ranks are then used to to an Instant Runoff vote.

Anyway... there are *much* better systems out there than the one you are planning to use, and I'm afraid you might be creating more problems than you're trying to solve...
 

It's not the winning...

But the being nominated that counts.

Personally speaking, any product that has been nominated (or even honourably mentioned) is enough for me to consider looking at it seriously as a potential purchase, and the knowledge that our esteemed panel of judges have looked at everything with a critical eye is sufficient for me.

I am, however, extremely undemocratic, and don't believe that hoi polloi should be allowed any form of voting at all. Ever. And that especially goes for stupid people at election time... :p
 
Last edited:

First off, as Greg said, I'm thrilled to have some of my work nominated. My discussions here are in no way related to my desire to win. :D

ColonelHardisson said:


The methods Morrus has used to try to make everything fair haven't been about eliminating the influence of popularity, exactly, but rather the influence of familiarity. That is, companies that can afford big marketing campaigns will be familiar to voters, while small companies will never have been heard of. What happens is that people will vote for something they've heard of, sight unseen, over something they haven't heard of, sight unseen. The current method attempts to address that, to give the small companies a chance.

The current method will only exacerbate the problem of familiarity, not provide relief. Are there going to be voting guidelines established (i.e. "If you haven't ever seen a product, then give it a 5.")? People will interpret this system in a myriad of ways that will skew the results away from what they intended, whether they are guilty of min/maxing the vote (:D) or just misinterpreting the "rules of the game."

ColonelHardisson said:

When it comes right down to it, though, the vote relies upon people voting their conscience.

I find this interesting because it is the opposite of what Psion said, which is "we should not second guess the motives of the voter." Your statement INSISTS that the voter have a perfect motive, otherwise the system goes awry. The legitimacy of an award should not hang its hat on the purity of voter motive. :)

Also, even the purest of motives would have a hard time understanding what to do in the case of a product they have not seen. It's easy to just not choose that product in a binary vote, but in this type of scheme giving such a product a "0" hurts it much more than simply not voting for it.

This can be somewhat mitigated if the default setting for each product is "I Don't Know." I hope that's the case.


I really wish the actual votes (or at least vote numbers, specifics aren't necessary) would be available after the process for examination. I think it will bear out what the cautioners are saying.
 

As long as every voter clearly understands the difference between a "0" and an "I don't know", then things should be fine.

As a market researcher, I have found that a 5 grade ranking scale is the most widely accepted, however. This is a minor point though.

Data "cleansing" is another typical part of the market research process. In fact many surveys purposely let you answer ( the same ) questions in an inconsistant manner so you can discount any one trying to fudge ( or just being careless ) with the survey.

Voters should realize that if they answer 10/0/0/0/0 then their vote will NOT be counted, as this is obviously bad data to input into any survey results. 10/2/2/2/2 should be discarded too etc...

10/DK/DK/DK/DK would be perfectly acceptable however.

Generally you will need some type of Ecxel ( or Access etc... )
data tabulation to compile results.

Example: results will look like this after tabulation and cleansing:

Best Dealy-wig I use all the time:

Product A: Total Votes: 940
Total Points: 7050
Average Score: 7.5

Product B: Total Votes: 329
Total Points: 2698
Average Score: 8.2

Product C: Total Votes: 551
Total Points: 4353
Average Score 7.9

Product D: Total Votes: 441
Total Points: 3837
Average Score 8.7

Product E: Total Votes: 767
Total Points: 6136
Average Score 8.0

So Product D, while perhaps only 3rd in the regognition/familiarity ( not "popular" ) vote, wins with a superior average rating.

The only problem you have in this type of voting is if you do not get a large enough vote for any one item in any category. Market research guidelines stipulate that you need at least ( about ) 200 inputs for any question to get above an 80% confidence rating ( 450 responses gets you 90% -/+ 5% )

-BFG
 

Remove ads

Top