MetaPost comments and questions about the messageboards and other parts of EN World. If you have a problem, this is where to go. Moderator email addresses are posted here.
You asked for some ideas on how to calculate the score for the reviews page. Well, here is an idea... Have the final score of the product calculated from the following formulae:
V x (A-1)^2
where:
V is the number of votes that the product has received
A is the average vote that would be displayed now
In other words, multiply the number of votes that the product has received by the square of the average vote minus 1.
By using this formulae, you will want to limit the to 10 lists and bottom 10 lists to those that have recieved at least 5 (or the current 7) votes. If you were to use this, you might want to display average vote as well.
what is the advantage of this system? I have not tested it much, but it seems to allow items with lower averages to beat out items with higher averages by getting more reviews. For example, the player's handbook would crush the manual of the planes with your system (if I am calculating it right), just by having more reviews.
Oh, now that something needs 7 reviews to get on the lists, how about something like a "rising stars" list, for the best products that haven't gotten that many reviews yet?
__________________ "A sound mind in a sound body"
-Juvenal
I'm sure it's a great idea, but I don't understand it. Could you explain how it works (the forumula that appears to be the much sought after unversal equation, that is, not the idea for a new list)?
If I understand your intention correctly, Morrus, you don't want to see a product with just 3 reviews of 5 topping a list, where the next in line has an average of 4,5 with 10 reviews. But on the other hand you do want to see a product with 6 perfect scores in a list where the top has an average of 3,8 with 10 reviews.
What about the following:[list=1][*]Each product is assigned a number, which is the sum of all review scores.[*]Only products, whose number from 1. is higher than a fixed value (e.g. 27), qualify as possible top 10 candidates.[*]From the list of possible candidates the final top 10 list is generated by ranking the average values.[/list=1]
This way a product with 6 perfect reviews could come into the list, as well as a product with 10 reviews and an average of 3.
On the other hand a product with just 4 reviews could not come into the list, even if all of them were a pefect 5.
__________________ - Stand back! I am an evil Acota.
- North Acota or South Acota?
To describe how it works
1) First of all, unless Liquide used some weird method when creating the reviews system, every products have 2 fields in the database, "total number of votes", and the "total value of all votes". Lets say I vote a 5 on Manual of the Planes, then the system adds 1 to total "total number of votes", and 5 to the "total value of all votes".
So, after a while, a product may have 12 votes and a total value of 48. If 12 people have voted the "total number of votes" will ALWAYS be 12, while the "total value of votes" will wary from 12 (if all voted 1) and 60 (if all voted 5).
Now we have two values. If we divide the total value of all votes with the total number of votes, we have the average rank of the product (eg 48/12 = 4)
2) Now, when we know the average, lets do as KDLadage described: "In other words, multiply the number of votes that the product has received by the square of the average vote minus 1. "
This means, simply put, that a product with a whole lots of reviews may be equally worth as a product with a few but high ranks.
As an example, look at this. We have book 1 where 4 people voted 5, and we have book 2 where 20 people voted 4.
book 1: 4 x (5-1) ^ 2 = 64???
ehh...
KD.. Have I missed something here?
__________________ The river sparkles and dances on the rocks.
At night, it flows deep and strong.
Originally posted by Psionicist The final equation will then be
V x (T / V - 1) ^ 2
Using the standard and KDLagade equations
V = Number of votes
T = Total pool of ranks
To describe how it works
1) First of all, unless Liquide used some weird method when creating the reviews system, every products have 2 fields in the database, "total number of votes", and the "total value of all votes". Lets say I vote a 5 on Manual of the Planes, then the system adds 1 to total "total number of votes", and 5 to the "total value of all votes".
So, after a while, a product may have 12 votes and a total value of 48. If 12 people have voted the "total number of votes" will ALWAYS be 12, while the "total value of votes" will wary from 12 (if all voted 1) and 60 (if all voted 5).
Now we have two values. If we divide the total value of all votes with the total number of votes, we have the average rank of the product (eg 48/12 = 4)
Why the heck would we *ever* want to do something like this? More votes outweigh (and have the potential to vastly outweigh) competing, higher-tranked products. For example, the #1 ranked Manual of the Planes would fall prey to the lower-ranked Player's Handbook - run the numbers!
I don't know why KD and Psi think this is a good idea, but I certanly don't think so. I'm comfortable with formulae, but this is absurd.
(If this was a message board for engineers, I'd call this a troll. )
Last edited by CRGreathouse; 3rd February 2002 at 02:50 AM..
...what it was was an attept to measure the ammount of "buzz" that a product generates as well as the initial reaction to that product.
For example, if a book generates 20 reviews, all told with an average of 4, I (personally) hold this in higher regard than a product with an average rank of 5 with only three votes after being on the shelves for a year...
Some of this is fixed with the idea of waiting for 7 votes, but look at the "bottom 10" listing -- some of this have "above average" ratings... because the pool of books with 7+ votes is much slimmer... This again, is one way of keeping the "three-vote wonder" out of the running.
I was simply offering an alternative. There are others that could be employed. For example, since the database keeps a running tally on every product, you could do something like this:
(V/P) x (A-3)
Where V is the number of votes the product received; P is the average number of votes each product in the database has received; and A is the average rank of teh product itself.
This would give a slight buzz factor, cause low ranking votes to push the product down (3=0, lower than 3 is negative, etc...)
First, KD, I'd like to apologize for being unkind, and thank you for not replying in the same tone I wrote in. I didn't mean to be caustic.
The new system is certainly an improvement over the last, as more reviews for a bad product give it a worse rating insetad of a better rating. It only works when reviews are actually centered on 3, and even then the more popular books hold a major advantage - a bigger advantage than their score, in many cases.
I will propose a system myself, despite my misgivings about doing so. A 90% confidence interval test could be used for rankings. The more people that review the product, the closer the intervals get:
10 votes, average 4.3; 90% confidence of a score from 3.4 to 4.6
20 votes, average 4.3; 90% confidence of a score from 3.6 to 4.5
I just made these numbers up, BTW. The ranking system would take the lower bound as the ranking.
This test would be much more logically sound, but I have to admit that despite being a mathematical hobbyist, I don't know how to calculate confidence intervals. Anyone who knows about statistics and normal distribution is welcome to chime in...
Until then, though, I think we're best off using a simple average.
Why is noone (besides number 1) really interesed in the tennis world ranking? Because noone can follow the formulas to calculate it.
I think it would not make the ranking tables very interesting, if they are sorted according to numbers, and nobody can follow the calculation system.
I'd say anything nonlinear is too complicated.
(I'm a mathematician, so I'm really not afaid of formulas in general )
The ranking should be done by the average, and nothing else.
Who qualifies for the top 10 - on the other hand - should be a little more complicated than just a minimum number of votes. Perhaps my suggestion above, perhaps something a bit stronger here...
I also do not want to see the PHB topping the list...
The top 10 should measure only quality, not amount of 'buzz'.
__________________ - Stand back! I am an evil Acota.
- North Acota or South Acota?
I am still trying to figure out why people want a different system other than strait averages. Why do number of reviews matter? Just because more people review a product doesn't mean it is a better project. I see the whole case for needing 7 reviews to make the top boards, but I really don't like that either. While requiring 7 reviews makes sense, nothing was done for those with less than 7 reviews, like a board for the top products that don't have that many votes yet. Something like products to watch, especially in the adventure area, where fewer people own the adventure (because of sheer numbers and how many people use pre made adventures) and thus there are less reviews.
To be honest, I never really liked the set up of the reviews page. Its great if you want to buy the hottest products, or already know what you want to look up, but what about for finding a product you never heard of? Personally, I would prefer a full list of products or a few full lists of different types of products, based on the scores of those products. That would make finding new products easier for me at least, instead of having to look through each publisher.
Just my thoughts on the whole reviews area, don't know if anyone else feels the same way.
Just my thoughts on the whole reviews area, don't know if anyone else feels the same way.
__________________ "A sound mind in a sound body"
-Juvenal
Thus, a product with a mean and mode of 3 each would have a rating of 3.00
However, the use of MODE in the equation would keep a single outlier from shifting the equation too much. In the case of a product with multiple modes, the average of the modes would be used...
Originally posted by KDLadage What about something like this:
(MEAN * MODE) / 3
Mean would have a mid-point of 3.
Mode would also have a mid-point of 3.
Thus, a product with a mean and mode of 3 each would have a rating of 3.00
However, the use of MODE in the equation would keep a single outlier from shifting the equation too much. In the case of a product with multiple modes, the average of the modes would be used...
Why mode instead of median? Median tends to be much more meaningful... 5 votes for 3, 4 botes for 4, and 3 votes for 5 is closer to 4 (median) than 3 (mode).
Second, (Mean*Mode)/3 is less meaningful than Sqr(Mean*Mode), the geometric mean. In fact, Sqr(Mean*Median) is even more meaningful...
Still, none of this is likely to matter, because I doubt the exact scores were kept, in favor of a "running total" as Psi stated. Thus, median and mode are *gone*.
Cool. OK, then, if Morrus likes the system, it's all ready, as far as I can see.
I still don't understand it...
I get the mechanics of it - and could perform the calculation no problem. I don't get the logic behind using it, though (although I'm sure your logic is sound).
Can someone explain how it works, and its benefits, in layman's terms?
I don't know exactly, but running it on a few items, the scores are fairly close to just the mean. I think the main purpose is to give the consistancy of the reveiws a slight importance. If you have a 3 given to a product with lots of 4s and 5s, or a product that gets a 4 while having mostly 2s and 3s, that extreme reveiw gets less influence, because while the mean dropped because of it, in most cases the median did not.
I have one really bad gripe though. The Forge of Fury as the tenth worst product, with a 3.78. Thats a very good rating considering the 10th best products are weighing in around 4.0. Is seven reveiws really the right number, especially for worst products? I mean, how many people are going to buy a product with a 2 rating and reveiw it, I doubt 7 people. I don't know, but 5 reviews seems a better number to me.
__________________ "A sound mind in a sound body"
-Juvenal
Originally posted by Morrus Can someone explain how it works, and its benefits, in layman's terms?
It decreases the effect of a single extreme vote, while still letting it count for something. Example:
Votes: 3, 3, 3, 4, 4, 4, 4, 4, 5, 5
Old method: 3.90
New method: 3.95
Votes: 1, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5 (same, with an extra vote of 1)
Old method: 3.64
New method: 3.81
When the votes are close (as in the first example), the two methods have similar results, but when one stray vote is included, this proposed method minimizes its effect on the total score. This works both ways - a stray high vote is also minimized.
All of this is transparent to the user; the new method is still an average, though not a "typical" average. The results are i the same range as the original vote.
)
)
Linear Mode
And yet another word from our sponsors
