Redesigning the Reviews Page

[drnuncheon]

Weighting reviews/reviewers

Here's my suggestion: allow people leaving comments about a review to rate the reviewer on the same scale, in a "how helpful was this review" box.

This makes the entire thing basically self-correcting. If a person puts out a bunch of crummy reviews that amount to nothing more than "I like this product" or "This product sucked" then their opinion will be marked down and count for less.

If they put out a series of decent, well-thought-out reviews that genuinely help people decide on whether or not to purchase the product, they will get marked as 'more helpful' and the weight of their opinion will rise.


J

 

[drnuncheon]

This is in response to some suggestions by Charles and Psion:

I ran some numbers on the idea of minimizing aberrant ratings by using sqrt(mean*median) and I don't like what I see, especially when dealing with extrme values. Using this method actually increases the effect a single aberrant review has on a product.

Consider this set of ratings: 5, 5, 5, 5, 1

Using the arithmetical mean (that's "average" for folks who don't speak statistics) you get a rating of 4.2.

The median (halfway between the lowest & highest points) is 3.

The sqrt(mean * median) is 3.54. The single dissenting review has pulled the score down even lower under this method than it does under the current one.

Heck, if you had 99 5s for a product, and some joker put in a 1, the rating would go from 5 to 3.85 (presumably rounded to 4), where just using the average would give you 4.96 (presumably rounded to 5) - that means that 1 single review out of 100 pulls the score down by more than a point! I don't think that makes for an accurate portrayal.

Now, this may have less effect where the scores are less extreme, but the effect is there - putting in a markedly lower (or higher) score drastically alters the median, and since you're giving the median as much weight as the mean in the final score, you're greatly multiplying the effect of that single review.

J

 

[Warchild]

Well, i don't know jack crap about medians and mean and such.
I like the current scoring system. It seems more honest to me.
As for the "spikers" several people have mentioned, i have this to say:
If i see several 5's and a 1, I know that there was a fruit-loop rating thrown and i can ignore it. What i actually do is take a look at the reviewer's history and judge whether i should ignore the guy or not. Thats what that whole set up is for isn't it??
I don't need ENWorld or a Einsteinian math system to make up my mind for me. Let me think for myself, please.

As for weighting reviewers, what a terrible idea. What you want is MORE people doing reviews, thats how you'll get the right averages. But if some people's reviews become weighted, then most people that might submit a review, might not when they find that their opinions count for less. Some of us that are writing reviews will stop. i will. I have nothing against Alan and Simon(do you have a reviewer hidden away somewhere called Theodore?), but no one's review should weigh in more than anyone else. That way lies madness and conspiracy theories!!

I like the Rating system as it is. It forces people to make real choices, is the product good enough or isn't it? The rating system of 1 to 5 carries more weight and impact than 1-10 or even 1-7. I think a larger scale only creates dodges for people wanting to be liked or create joy-joy fuzzy feelings, instead of just giving the product a 2, if it darn well deserves a 2!! If a product isn't a 4 then its a 3. Stick to your guns or get out of the armory!!

Call me a hardcase if you will, but thats how i see it.

 

[Furn_Darkside]

3) Don't use colored text. It doesn't work for everyone, i.e. those with vision problems. Currently there is use of red on "red" in the reviews pages. That's hard to read for everyone, I imagine.

4) Don't mess with font type or size. Allow user's to control that with their browsers.

Salutations,

First- let me state how much I appreciate the review pages- they are a great resource.

However, I agree with Corvus- especially with the current reviews being hard to read. Grey text on a black background is tiring on the eyes for the longer reviews.

Thanks for the hard work.

FD

 

[KDLadage]

...a quick lesson in statistics...

I ran some numbers on the idea of minimizing aberrant ratings by using sqrt(mean*median) and I don't like what I see, especially when dealing with extrme values. Using this method actually increases the effect a single aberrant review has on a product.

Consider this set of ratings: 5, 5, 5, 5, 1

Using the arithmetical mean (that's "average" for folks who don't speak statistics) you get a rating of 4.2.

The median (halfway between the lowest & highest points) is 3.

Incorrect.

In this case the mean is 4.2 (5+5+5+5+1=21; 21/5=4.2)

The median is the central value of a sorted set. With 5 values, sort them and select the 3rd value -- in this case 5.

Thus 4.2 x 5 = 21; sqrt(21) = 4.58

 

[Nail]

"Newbie" take

Just so y'all know, I haven't read a darn thing in this thread. So think of this as just "extra data", rather than unique suggestions.

I jus' discovered the reviews page a week or so ago, as only recently have I decided to squeeze out some o' th' hard-earned cash. What I wanted on a review page was:

- Five star ranking system, sortable by company or catagory, with rankings only given to those products with many (read: more than, say, 10) reviews. Heck, all I need is the number, so if people jus' wanna put a rating down an' be done w' it,...well that'd be fine. Suggestion: develop an easier "vote" system to collect more voters, so people don't feel they have to write a Magi's tome to review somethin'.

- Li'l snippets of some of the written reviews on a summary page. Ya got that. Good, but how about some way to sort them? Like: "Hey, give me all the reviews that had a 5 star rating for this product. Good. Now give me all the reviews of people that hated it. Excellent...."

-Links from the reviews to a few places that sell the product, plus the publishers web site.

That's it, really. Anything else is gravy.

-Nail

 

[drnuncheon]

Re: ...a quick lesson in statistics...

Incorrect.

In this case the mean is 4.2 (5+5+5+5+1=21; 21/5=4.2)

The median is the central value of a sorted set. With 5 values, sort them and select the 3rd value -- in this case 5.

Thus 4.2 x 5 = 21; sqrt(21) = 4.58

Whup, my bad. You are right. This makes a lot more sense than what I thought you were doing.

Carry on, then.

J

 

[drnuncheon]

As for weighting reviewers, what a terrible idea. What you want is MORE people doing reviews, thats how you'll get the right averages. But if some people's reviews become weighted, then most people that might submit a review, might not when they find that their opinions count for less.

Note that their opinions would only count for less if most people on the server thought their opinions should count for less. (Then again, my rating idea isn't based on the number of reviews like some others are. Just because you write a lot of reviews doesn't mean they're good - I'd rather judge quality, not quantity.)

All it's doing is exactly what you do when you see a reviewer and say "Oh, I know them, they never think things through when they review" - it's just reflecting that in the scoing.

Otherwise you have John "I giv this a 5 cuz it has nekkid girl in it" Smith's rating counting exactly the same as anyone who gives a serious, worthwhile review.

Of course, the ideal would be one like Alexandria Digital Literature(www.alexlit.com) has - it matches your ratings up with the ratings of everyone else in the DB to predict how much you personally will like a product. Dunno if Morrus wants to do anything that complex though!

J

 

[Moon_Goddess]

Morrus know's I only have one request.

Make the text size resizable, I never use the reviews on the site as it stands now becuase I would have to change my screen resolution, to do so. Control- MouseWheel, is the greatest thing to happen in computing in years. Please don't turn it off.

 

[Psion]

Re: more

Original Content
Presentation
Rules accuracy
Your take

And then average the 4 results, either straight up, or some kind of weighting system we come up with later?

I'm not so sure how productive that would be. Although it would be nice to think that we can get some reviewers to review using similar criteria, you really can't, and most people aren't going to have a firm enough view of a product to make a meaningful assessment of each quality. When I am trying to rate a product, in my head I give it a +, 0, or - in 5 areas*, 2-3 is a 4 and 4+ is a 5... but even I think my opinion isn't so niggly I could put a 1-10 number on 4 qualities.

However, I agree that there have been many times I had wished I could assign half points while just using one number.

* - in case you care, those are presentation, quantitative value, system fidelity, ideas, usable material.