Li Shenron
Legend
The thread is old, but the topic is very actual.
I think reviews (of everything) and generally a really good thing, they allow us to make much more informed purchase decisions than we used to do before the internet. Whether you're buying a car, a coffee machine, or a D&D adventure, it's definitely a good idea to read some reviews first.
But then, you need to be able to parse and weight those reviews. Know where to look for trusted or professional reviewers, but be aware that even those need to be read properly. The bigger the purchase, the more informed you should be, so generally I wouldn't say to spend that much time for a D&D adventure compared to when you buy something more onerous. I definitely have no interest for "shiny and new" purchases, to the point that I scorn people who queue for hours outside a shop to be among the first to buy the latest mobile phone or something like that.
That said, the vast majority of people do not know how to write a review. They buy something and immediately want to share their feelings, but what they are really sharing doesn't help others if they haven't used what they bought.
The primary difficulty with reviewing a D&D adventure (or a computer game, or a RPG product in general) is that they can be played in many ways, that's the beauty of a RPG book! Even if you follow the adventure "by the book" as much as possible, no two groups will experience the same thing, and if an adventure doesn't work for a group it doesn't mean it's the adventure's fault. In theory, the more playtest reviews are written the more reliable the overall picture, because some DMs might have failed at running the adventure, a few others might have had a fantastic time only because their DM are awesome anyway, but the extreme cases balance each other out eventually.
Unfortunately, a read-only review is worth much less, although it's not completely worthless, because the reviewer can only imagine how the adventure will be played out: at best, they can have a useful opinion on the general plot and the editorial qualities of the book (layout, text, art...) which is still useful information but doesn't tell you how it really works. It has the same value of all those mobile phone reviews written on the same day of the purchase, with the reviewer all hyped up about their new shiny toy, but they won't tell you that the iCrap screen will be scratched after a week, the battery needs to be charged twice a day, and the memory will be already full after you install a couple of more apps.
The best thing to do with a RPG, unfortunately for the publishers, is wait... and wait... and then wait a bit more... After a few months since publication, then you will really start seeing people who actually played the adventure / used the book, and you can get a better idea by seeing what is the most common opinion, even if you just collect comments rather than full reviews.
I think reviews (of everything) and generally a really good thing, they allow us to make much more informed purchase decisions than we used to do before the internet. Whether you're buying a car, a coffee machine, or a D&D adventure, it's definitely a good idea to read some reviews first.
But then, you need to be able to parse and weight those reviews. Know where to look for trusted or professional reviewers, but be aware that even those need to be read properly. The bigger the purchase, the more informed you should be, so generally I wouldn't say to spend that much time for a D&D adventure compared to when you buy something more onerous. I definitely have no interest for "shiny and new" purchases, to the point that I scorn people who queue for hours outside a shop to be among the first to buy the latest mobile phone or something like that.
That said, the vast majority of people do not know how to write a review. They buy something and immediately want to share their feelings, but what they are really sharing doesn't help others if they haven't used what they bought.
The primary difficulty with reviewing a D&D adventure (or a computer game, or a RPG product in general) is that they can be played in many ways, that's the beauty of a RPG book! Even if you follow the adventure "by the book" as much as possible, no two groups will experience the same thing, and if an adventure doesn't work for a group it doesn't mean it's the adventure's fault. In theory, the more playtest reviews are written the more reliable the overall picture, because some DMs might have failed at running the adventure, a few others might have had a fantastic time only because their DM are awesome anyway, but the extreme cases balance each other out eventually.
Unfortunately, a read-only review is worth much less, although it's not completely worthless, because the reviewer can only imagine how the adventure will be played out: at best, they can have a useful opinion on the general plot and the editorial qualities of the book (layout, text, art...) which is still useful information but doesn't tell you how it really works. It has the same value of all those mobile phone reviews written on the same day of the purchase, with the reviewer all hyped up about their new shiny toy, but they won't tell you that the iCrap screen will be scratched after a week, the battery needs to be charged twice a day, and the memory will be already full after you install a couple of more apps.
The best thing to do with a RPG, unfortunately for the publishers, is wait... and wait... and then wait a bit more... After a few months since publication, then you will really start seeing people who actually played the adventure / used the book, and you can get a better idea by seeing what is the most common opinion, even if you just collect comments rather than full reviews.