Redesigning the Reviews Page

[War Golem]

My essential question is - if it is even possible, is it "bad form" to simply review a product by giving it a score, rather than a full-blown written review?

I have never reviewed anything here at EN World, so I am not intimately familiar with the procedure for doing so. Nonetheless, I certainly have a lot of d20 material, and would be happy to score a lot of it to even out the average rating and bump up the number of reviews per product (and thus hopefully help out other board members with their purchases)... but there's simply no way I have the time to write actual formal, written reviews of all this stuff.

Please advise, Morrus or Psion or anyone who deals with the EN World reviews.

Thanks.

 

[Morrus]

My essential question is - if it is even possible, is it "bad form" to simply review a product by giving it a score, rather than a full-blown written review?

I have never reviewed anything here at EN World, so I am not intimately familiar with the procedure for doing so. Nonetheless, I certainly have a lot of d20 material, and would be happy to score a lot of it to even out the average rating and bump up the number of reviews per product (and thus hopefully help out other board members with their purchases)... but there's simply no way I have the time to write actual formal, written reviews of all this stuff.

Please advise, Morrus or Psion or anyone who deals with the EN World reviews.

Thanks.

It's discouraged. A simple score is not very useful to people - they need to know why you gave it that score. Requiring an actual review also encourages people to be more thoughtful about the score they give.

When the review page was orginally created in the early months of 2001, it was possible to just rate a product without scoring it. The feature was immediately and widely abused and was quickly removed. I have no intention of putting it back.

 

[Erithtotl]

more

I have no problem with using the root(mean*median) method. It seems like this would make sense for the most part. I was also thinking of converting the ratings to a 1-10 scale instead of the 1-5 that we have now. I'll have to run this by Morrus but what it will in-effect do is allow for halves (3.5 = 7). I'll just throw this out as a possibility: What about rating multiple features about a product, and building in some weights to those ratings. For example:

Original Content
Presentation
Rules accuracy
Your take

And then average the 4 results, either straight up, or some kind of weighting system we come up with later?

The only thing about not allowing the option of rating without reviewing, is couldn't people just put in nonsense for their review as an opportunity to rate it? I kinda like the idea of forcing a review, but there's only so much we can do. Another option is allow people to actually rate the reviews. If there are more than x number of ratings (say, 5) for a review, we then use that rating as a weight on the review rating. The idea is that if many people disagree strongly with a review, they can rate it poorly, eventually reducing its effect on the overall average product rating. It sounds a little complex but it shouldn't be too hard to implement, and it might solve several problems.

Ian

 

[KDLadage]

some thoughts...

I have no problem with using the root(mean*median) method. It seems like this would make sense for the most part.

I agree. I think it would solve a great number of issues with the reviews.

I was also thinking of converting the ratings to a 1-10 scale instead of the 1-5 that we have now. I'll have to run this by Morrus but what it will in-effect do is allow for halves (3.5 = 7).

Should not be too much of a leap of faith to do that one. I would let all of those that have reviews in the datbase know so that they can go and modify thier review scores to the +/- 1 values if need be.

I'll just throw this out as a possibility: What about rating multiple features about a product, and building in some weights to those ratings. For example:

Original Content
Presentation
Rules accuracy
Your take

And then average the 4 results, either straight up, or some kind of weighting system we come up with later?

I would say no.

I would say no for several reasons: many people will place widely varying weights on each catagory you rate the product on. Leave the rating as a single score; let the review tell the tale.

The only thing about not allowing the option of rating without reviewing, is couldn't people just put in nonsense for their review as an opportunity to rate it? I kinda like the idea of forcing a review, but there's only so much we can do.

I would not worry about it. As it is, we have nearly 800 reviews, and nobody has really abused the system that I can tell.

Another option is allow people to actually rate the reviews. If there are more than x number of ratings (say, 5) for a review, we then use that rating as a weight on the review rating. The idea is that if many people disagree strongly with a review, they can rate it poorly, eventually reducing its effect on the overall average product rating. It sounds a little complex but it shouldn't be too hard to implement, and it might solve several problems.

I think it causes others, however.

The only reviews I would give greater weight to would be those of the "official" reviewers. I do not say this because they are "official" butbecause they are the most well thought out, well written and well recieved reviews in the database, for the most part.

 

[corvus]

1) Add a simple search on the main page. Have it match on any of title, author, or publisher by default, but allow user's to change the default with a pop-up. See how Amazon.com does it.

2) Use tabs to navigate between the main areas of the reviews section. Again, see Amazon.com.

3) Don't use colored text. It doesn't work for everyone, i.e. those with vision problems. Currently there is use of red on "red" in the reviews pages. That's hard to read for everyone, I imagine.

4) Don't mess with font type or size. Allow user's to control that with their browsers.

5) Simplifly the main page. There's just too much stuff there now which makes it hard to find what you want.

6) On the publisher pages, make the product's title a link to that product's reviews.


Also it would be nice if the link to the reviews page was more prominent on the enworld.org top page, like where it currently says:

D&D/d20 System News & Reviews

at the top. Reviews could link to *gasp* the reviews page.

 

[War Golem]

It's discouraged. A simple score is not very useful to people - they need to know why you gave it that score. Requiring an actual review also encourages people to be more thoughtful about the score they give.

When the review page was orginally created in the early months of 2001, it was possible to just rate a product without scoring it. The feature was immediately and widely abused and was quickly removed. I have no intention of putting it back.

Aya, aye captain. No reviews without body text!

Thanks for the edification.

 

[CRGreathouse]

Re: more

I was also thinking of converting the ratings to a 1-10 scale instead of the 1-5 that we have now. I'll have to run this by Morrus but what it will in-effect do is allow for halves (3.5 = 7).

From what I'v read, rating from 1-10 is too much for most people to do accurately - 1-7 has been called the "ideal" system, as it gives enough choice, but not too much. I stress that I don't know this to be true, though it makes sense; I'm just bringing it up for discussion.

As for the weighting system, I'll explain a little more. Let's say a person is willing to "work the system" with an unpopular product s/he likes. The person wants it to be rated a 4, but the average is now 2. Obviously, the person would rate it a 5, bringing it closer to 4 than if s/he had rated it a 4.

With the median, there's no reason to overshoot - 5 won't being it any closer than 4.

Some would object to the median, though, because it means that extreme reviews count for nothing, and that most or all reviews will have a whole-number rating. Thus, the geomean of the two methods provides an excellent solution to the problem.

 

[Suzerain]

***

 

[bramadan]

I am not a big fan of median but agree that the idea of having outliers count for 50% is probably decent (if something like that is to be done). I personaly like the 7 reviews+ rule as it keeps the top lists to established classics. What would be a good thing to have to offset this is "top 10 new releases" list that would require only 2 or 3 reviews but products could stay on it only month or two after they are released.
I also second 1-10 scale, I do feel there is noticable quality difference between some stuff I have to give same mark to.
Merging WotC and d20 list is also a good idea.

Also on a personal note, Morus or Erithtotl, if you could merge my reviews under Bramadan and Bojan (bramadan) it would be great. I have lost the pasword for the old account and have started the new one a while ago but would like to have them all in one place (especialy if the number of reviews is going to count for anything). I mailed Morus about this a while ago but he was probably too busy....
Thanks a lot.

 

[Clint]

I'm voting for a smaller page template.

My biggest problem with the reviews page is that the server seems to be swamped at many points during the day.

Instead of listing all the companies and reviewers and the Top 10's on every page, simplify the design. I don't look at that information every page view, but I download all of it, every time, and it brings the size of the page way, way up. (to ~120KB, not counting graphics)

I'd much rather have a list of three simple links to cover those three topics. The left bar could look like this:

link:Main Page
---
Staff reviews by:
link:Simon Collins
linksion
---
link:Top 10 Lists
---
link:Search by Publisher
link:Search by Reviewer
Search by Product Name:
[form]

I saved the front page locally and cut out the sections I mentioned. You could quickly trim the page down to 20KB if you replace those sections with links. Easier on the server, easier on the users.

Keep the "most recent reviews" expanded on the page, though.

Also, a slim/no-graphics option (stored in a cookie) would be pretty sweet.

Good luck, I can't wait to see the results!

-Clint