ENnies - let's launch the voting booth!

[Wulf Ratbane]

Ah, that'll explain the single vote for HoHF...

So yer saying I have a perfect 10 score! Sweeeeeeet!


Wulf

 

[Morrus]

So yer saying I have a perfect 10 score! Sweeeeeeet!


Wulf

But it's disqualified for not meeting the 100-votes minimum threshold.

 

[coyote6]

I know, I was just giving you a hard time.

This, of course, is why the "certain voting patterns" that are going to be discarded can best be described as "high votes for books that Upchurch guy worked on."

 

[d20Dwarf]

This, of course, is why the "certain voting patterns" that are going to be discarded can best be described as "high votes for books that Upchurch guy worked on."

Yeah, well, luckily they won't have much work in that case.

 

[Olgar Shiverstone]

If you don't mind me picking your brains a little further - what, specifically, would you suggest we do with the figures we have? Bear in mind that we're already doing two of the things your suggested (removal of obviously biased voting patterns and cut-off levels for minimum numbers of votes).

I'd be interested in seeing the exact formula you would apply to the info we have. We know each individual vote, we know the number of votes for any given product, and we know the average score (the mean in this case). I'd imagine that Blacksway could easily run the numbers through a different calculation routine if it were explained simply, step-by-step.

OK, given that you're already dropping outliers and using a minimum cutoff score, at a minimum what I'd recommend you do next is normalize all the scores by individual voter. Here's how:

1. Take the average of all of an individual voters scores (Sum of scores/# of scores -- ignore "not familiars").

2. Calculate the sample standard deviation for that individual's scores (easiest way to do this is with a function like STDEV in Excel, or I'm sure there's a similar capability in whatever tools Blacksway is programming in. There is a brute force method I can provide if need be).

3. Correct each individual score, using the formula:

(X - Mean)/STDEV, where

X- score
Mean -- average of that individual's scores
STDEV -- the standard deviation calculated above.

At this point you'll have converted all the scores into a range that will generally fall between -3 and -3, with the exact mean at zero. The number represents the number of standard deviations away from average the product falls on a standard normal (bell) curve.

Example (hopefully the formatting comes through)

Score Norm Percentile
4 -0.2397 41%
3 -0.6391 26%
9 1.7577 96%
2 -1.0386 15%
3 -0.6391 26%
4 -0.2397 41%
4 -0.2397 41%
8 1.3582 91%
7 0.9587 83%
2 -1.0386 15%

Mean: 4.6
STD DEV: 2.503331114


There's a choice here -- you can work directly with these scores, or convert them into percentile ranks (using a Z-table function, like NORMDIST in Excel, which I've used above). What you do depends upon how you want to make the final decision:

For example, the winner could be the one with the highest average score. In that case, sum all the normed scores for that particular product, and divide by the number of votes the product received (you could convert the to percentages before summing or after dividing, if you want a score between 0 and 100). This gives the win to the highest "quality" grade of the product, but the actual number of votes doesn't matter, so long as there's enough to meet the minimum requirement (more votes will mean a more accurate assessment of product quality, though).

Or, you could use the highest total score. This rewards a large number of votes, but takes into account the quality of a product In this case, sum the normed scores -- this is important, because the negative scores actually reduce the total, so it isn't possible for a product with all below-average scores to get a positive score, regardless of the number of votes it receives. It guarantees that the highest positive score had a majority of votes with at least "above average" quality. It won't actually select the "best quality" product based on ratings -- a product with a lot of "8's", for example, can beat a product with a smaller number of "10" votes. It does provide a balance of quality and quantity, though, and the normed scores won't reward below-average product ratings. OF course, the final number won't mean much to most people.

A third option would be to provide use a weighted decision criterion between the first method I mentioned and the second.

Not knowing exactly how you want the awards assigned, you'll have to consider the options. Overall I think the second method provides a reasonable balance that rewards both the popularity of a product (number of votes received) and the quality of product (level of scores).

As to running an ANOVA at the end, you really can get away without doing this just by looking at the scores and applying a little common sense. I suspect in the end that there will be fairly clear winners in each catagory; the ANOVA comes into play where one product wins out numerically by a second or third is very close behind. In any case, the ANOVA would require a dedicated stats package to run -- the full procedure is too long to detail here.

Food for thought.

 

[spacecrime.com]

I gotta say, I'm feeling all tingly about this voting thing!

Gonna spend a little more time this afternoon perusing the candidates, and then I'll be stuffing... er, I mean, visiting ... the ballot box this evening!

Morrus-san: I'm just being nosy, but it would be cool to occasionally hear a rough figure on how many ballots have been cast.

cheers,

 

[Crothian]

I'm amazed at some of the low numbers of people voting for the free stuff and the fan sites. I would have thought these would be the easiest for people to look at and form an opinion on.

So, if you haven't voted yet please take some time and look at the products you can like:

Best Campaign Fan Site
Best Resource Fan Site
Best Official Website
Best Art (Cover)
Best Free Product or Web Enhancement

All of the nominations in these five categories can easily been seen by following the links from here:

http://www.enworld.org/modules.php?...ns&file=index&req=viewarticle&artid=40&page=1

 

[Wulf Ratbane]

I'm amazed at some of the low numbers of people voting for the free stuff and the fan sites. I would have thought these would be the easiest for people to look at and form an opinion on.

Best Art (Cover)

All of the nominations in these five categories can easily been seen by following the links from here:

http://www.enworld.org/modules.php?...ns&file=index&req=viewarticle&artid=40&page=1

Yes, yes, please vote for the cover-- "not familiar" is no excuse! You can compare each of the covers, without bias of the contents, to any of the competitors. It doesn't get any easier than that!

My personal involvement in our cover is pretty minimal (concepting only) but Brad and Andy, the guys who work to make our covers "POP!" the way they do, workin' their mad Illustrator and Photoshop skills to add color, texture, and depth-- deserve every bit of recognition they can get!


Wulf

 

[Pramas]

If you haven't seen the gorgeous art in Freedom City, you can see some samples here:

http://www.drizzle.com/~kylo/fcgallery.html

To see a sample of the graphic design and layout, you can download this preview of Freedom City as well:

http://www.mutantsandmasterminds.com/files/FreedomCityPreview.pdf

To get the full effect, you really ought to at least page through the book at your FLGS (or, of course, buy it), but the above links give a nice taste of the final book.

 

[Pramas]

I posted a bunch more links for Green Ronin nominees in the Publisher's area. Thread is here:

http://enworld.cyberstreet.com/showthread.php?s=&threadid=56215