tleilaxu said:
Here is a suggestion that I just thought of that I think could make the ratings more accurate.
Ooh, my buttons are getting pushed
A thing like this does not increase
accuracy. It increases
precision. There's a difference.
Accuracy talks about how correct the number is, how well it reflects reality. Precision, in essence, talks about how many decimal places the measurement can realistically include.
A laser range finder can take measurements very precisely. It can always give you a number out to many decimal places. Your measurement may still be inaccurate if you use this very precise tool incorrectly (by saying you measured the width of a field when you actually measured it's length, for example).
There is no point to taking a measurement precisely if you aren't going to be very accurate anyway. That's the situation we have here. The rating system is subjective (since it's rating an opinion, and opinions are subjective). There's no "code" for how you give a rating. Unless everyones giving ratings in exactly the same way, for the same reasons, you're not getting a very accurate measurement. There's little point in going for high precision on it.
In other words, the ratings system is by it's nature very vague. Giving more info on the ratings is then not terribly useful. It actually then makes sense to keep the ratings vague, rather than giving percentile scores, or going to a 1 to 10 system. You actually have a greater chance of getting consistent accurate measurements if you give people less freedom of "false precision".
Also, how many ratings do most products get? Percentile scores are not particularly good if the number of scores is less than (you guessed it) 100 or so. If only 5 people rate a product, a percentile score is pretty meaningless. If there are many such products, and you weight for how many people actually rated the product, you now reduce many of the rating to statistical irrelevance. If you're throwing many of them away, what's the point of doing a percentile score in the first place?
In the end, it comes down to this - what do users want to get out of reviews? They want to know how good the product is. Percentile socres don't actually tell you that. They are a bit of statistical minutae that only tells you something about the products if their quality actually follows a normal distribution.
(edit: My typing and grammar are often rather inaccurate
)
