Calculation for the reviews page

[KDLadage]

Morrus,

You asked for some ideas on how to calculate the score for the reviews page. Well, here is an idea... Have the final score of the product calculated from the following formulae:

V x (A-1)^2

where:

• V is the number of votes that the product has received
• A is the average vote that would be displayed now

In other words, multiply the number of votes that the product has received by the square of the average vote minus 1.

By using this formulae, you will want to limit the to 10 lists and bottom 10 lists to those that have recieved at least 5 (or the current 7) votes. If you were to use this, you might want to display average vote as well.

 

[praetorian]

what is the advantage of this system? I have not tested it much, but it seems to allow items with lower averages to beat out items with higher averages by getting more reviews. For example, the player's handbook would crush the manual of the planes with your system (if I am calculating it right), just by having more reviews.

Oh, now that something needs 7 reviews to get on the lists, how about something like a "rising stars" list, for the best products that haven't gotten that many reviews yet?

 

[Morrus]

Umm... call me stupid - but, huh?

I'm sure it's a great idea, but I don't understand it. Could you explain how it works (the forumula that appears to be the much sought after unversal equation, that is, not the idea for a new list)?

http://www.hawking.org.uk/text/physics/physics.html

 

[bensei]

If I understand your intention correctly, Morrus, you don't want to see a product with just 3 reviews of 5 topping a list, where the next in line has an average of 4,5 with 10 reviews. But on the other hand you do want to see a product with 6 perfect scores in a list where the top has an average of 3,8 with 10 reviews.

What about the following:

1. Each product is assigned a number, which is the sum of all review scores.
2. Only products, whose number from 1. is higher than a fixed value (e.g. 27), qualify as possible top 10 candidates.
3. From the list of possible candidates the final top 10 list is generated by ranking the average values.
   [/list=1]
   
   This way a product with 6 perfect reviews could come into the list, as well as a product with 10 reviews and an average of 3.
   On the other hand a product with just 4 reviews could not come into the list, even if all of them were a pefect 5.

 

[Psionicist]

The final equation will then be

V x (T / V - 1) ^ 2

Using the standard and KDLagade equations

V = Number of votes
T = Total pool of ranks

To describe how it works
1) First of all, unless Liquide used some weird method when creating the reviews system, every products have 2 fields in the database, "total number of votes", and the "total value of all votes". Lets say I vote a 5 on Manual of the Planes, then the system adds 1 to total "total number of votes", and 5 to the "total value of all votes".
So, after a while, a product may have 12 votes and a total value of 48. If 12 people have voted the "total number of votes" will ALWAYS be 12, while the "total value of votes" will wary from 12 (if all voted 1) and 60 (if all voted 5).

Now we have two values. If we divide the total value of all votes with the total number of votes, we have the average rank of the product (eg 48/12 = 4)

2) Now, when we know the average, lets do as KDLadage described: "In other words, multiply the number of votes that the product has received by the square of the average vote minus 1. "
This means, simply put, that a product with a whole lots of reviews may be equally worth as a product with a few but high ranks.
As an example, look at this. We have book 1 where 4 people voted 5, and we have book 2 where 20 people voted 4.

book 1: 4 x (5-1) ^ 2 = 64???

ehh...

KD.. Have I missed something here?

 

[CRGreathouse]

The final equation will then be

V x (T / V - 1) ^ 2

Using the standard and KDLagade equations

V = Number of votes
T = Total pool of ranks

To describe how it works
1) First of all, unless Liquide used some weird method when creating the reviews system, every products have 2 fields in the database, "total number of votes", and the "total value of all votes". Lets say I vote a 5 on Manual of the Planes, then the system adds 1 to total "total number of votes", and 5 to the "total value of all votes".
So, after a while, a product may have 12 votes and a total value of 48. If 12 people have voted the "total number of votes" will ALWAYS be 12, while the "total value of votes" will wary from 12 (if all voted 1) and 60 (if all voted 5).

Now we have two values. If we divide the total value of all votes with the total number of votes, we have the average rank of the product (eg 48/12 = 4)

Why the heck would we *ever* want to do something like this? More votes outweigh (and have the potential to vastly outweigh) competing, higher-tranked products. For example, the #1 ranked Manual of the Planes would fall prey to the lower-ranked Player's Handbook - run the numbers!

I don't know why KD and Psi think this is a good idea, but I certanly don't think so. I'm comfortable with formulae, but this is absurd.

(If this was a message board for engineers, I'd call this a troll. )

 

[KDLadage]

It was not a troll...

...what it was was an attept to measure the ammount of "buzz" that a product generates as well as the initial reaction to that product.

For example, if a book generates 20 reviews, all told with an average of 4, I (personally) hold this in higher regard than a product with an average rank of 5 with only three votes after being on the shelves for a year...

Some of this is fixed with the idea of waiting for 7 votes, but look at the "bottom 10" listing -- some of this have "above average" ratings... because the pool of books with 7+ votes is much slimmer... This again, is one way of keeping the "three-vote wonder" out of the running.

I was simply offering an alternative. There are others that could be employed. For example, since the database keeps a running tally on every product, you could do something like this:

(V/P) x (A-3)

Where V is the number of votes the product received; P is the average number of votes each product in the database has received; and A is the average rank of teh product itself.

This would give a slight buzz factor, cause low ranking votes to push the product down (3=0, lower than 3 is negative, etc...)

Just ideas.

That is all they are.

 

[CRGreathouse]

First, KD, I'd like to apologize for being unkind, and thank you for not replying in the same tone I wrote in. I didn't mean to be caustic.

The new system is certainly an improvement over the last, as more reviews for a bad product give it a worse rating insetad of a better rating. It only works when reviews are actually centered on 3, and even then the more popular books hold a major advantage - a bigger advantage than their score, in many cases.

I will propose a system myself, despite my misgivings about doing so. A 90% confidence interval test could be used for rankings. The more people that review the product, the closer the intervals get:
10 votes, average 4.3; 90% confidence of a score from 3.4 to 4.6
20 votes, average 4.3; 90% confidence of a score from 3.6 to 4.5
I just made these numbers up, BTW. The ranking system would take the lower bound as the ranking.

This test would be much more logically sound, but I have to admit that despite being a mathematical hobbyist, I don't know how to calculate confidence intervals. Anyone who knows about statistics and normal distribution is welcome to chime in...

Until then, though, I think we're best off using a simple average.

 

[bensei]

Why is noone (besides number 1) really interesed in the tennis world ranking? Because noone can follow the formulas to calculate it.

I think it would not make the ranking tables very interesting, if they are sorted according to numbers, and nobody can follow the calculation system.

I'd say anything nonlinear is too complicated.
(I'm a mathematician, so I'm really not afaid of formulas in general )

The ranking should be done by the average, and nothing else.
Who qualifies for the top 10 - on the other hand - should be a little more complicated than just a minimum number of votes. Perhaps my suggestion above, perhaps something a bit stronger here...

I also do not want to see the PHB topping the list...
The top 10 should measure only quality, not amount of 'buzz'.

 

[praetorian]

I am still trying to figure out why people want a different system other than strait averages. Why do number of reviews matter? Just because more people review a product doesn't mean it is a better project. I see the whole case for needing 7 reviews to make the top boards, but I really don't like that either. While requiring 7 reviews makes sense, nothing was done for those with less than 7 reviews, like a board for the top products that don't have that many votes yet. Something like products to watch, especially in the adventure area, where fewer people own the adventure (because of sheer numbers and how many people use pre made adventures) and thus there are less reviews.

To be honest, I never really liked the set up of the reviews page. Its great if you want to buy the hottest products, or already know what you want to look up, but what about for finding a product you never heard of? Personally, I would prefer a full list of products or a few full lists of different types of products, based on the scores of those products. That would make finding new products easier for me at least, instead of having to look through each publisher.

Just my thoughts on the whole reviews area, don't know if anyone else feels the same way.

Just my thoughts on the whole reviews area, don't know if anyone else feels the same way.