D&D 5E D&D 5E: Ranking every published adventure, using Amazon reviews

[Zardnaar]

Hmm. Not terribly useful. I see 11/14 entries sitting at a 4.8 rating, with the outliers of that group within %5 points of each other. That doesn't help me narrow a choice down. At the best, and pretending Amazon reviews are all accurate & truthful, that just tells me WoTC produces a very consistent level of content.

Production values are very consistent.

 

[teitan]

What @Whizbang Dustyboots is getting at is that one can believe this and believe that Amazon reviews are not a good indicator of how well an adventure is received.

I think it’s quite obvious that both are true - the Tiamat adventures are not universally reviled. Indeed, they are quite well liked. But this data isn’t a reliable indicator of that fact.

Thousands of reviews aren’t an indicator?

 

[Whizbang Dustyboots]

Thousands of reviews aren’t an indicator?

No. There's an entire industry built around messing around with reviews, for reasons that aren't always obvious. (I don't think WotC is putting fake reviews into the system.)

This site suspects that 45% of the reviews on Hoard of the Dragon Queen are suspect, possibly to intentionally drive down reviews in a programmatic way.

The same issue happens whenever you let the public add reviews, most famously at Rotten Tomatoes.

 

[Charlaquin]

Thousands of reviews aren’t an indicator?

Not a reliable one. See Whizbang’s post.

 

[Swarmkeeper]

Would love to see a ratings poll of people who bought adventures from their LFGS - if that was a thing that was at all possible.

 

[Mistwell]

No. There's an entire industry built around messing around with reviews, for reasons that aren't always obvious. (I don't think WotC is putting fake reviews into the system.)

This site suspects that 45% of the reviews on Hoard of the Dragon Queen are suspect, possibly to intentionally drive down reviews in a programmatic way.

The same issue happens whenever you let the public add reviews, most famously at Rotten Tomatoes.

I found this fascinating so decided to re-run all of these using that tool. The results will take a while for it to update them all.

 

[Mistwell]

Product Name / (Pass Warn or Fail) / Adjusted Rating

Tomb of Annihilation (PASS) 4.7
Curse of Strahd (FAIL) 4.6
Out of the Abyss (WARN) 4.6
Tales from the Yawning Portal (WARN) 4.6
Dungeon of the Mad Mage (FAIL) 4.5
Ghosts of Saltmarsh (WARN) 4.5
Rise of Tiamat (FAIL) 4.5
Storm Kings Thunder (WARN) 4.5
Descent Into Avernus (FAIL) 4.4
Dragon Heist (FAIL) 4.4
Rime of the Frostmaiden (FAIL) 4.4
Hoard of the Dragon Queen (FAIL) 4.3
Princes of the Apocalypse (FAIL) 4.3
Candlekeep Mysteries (FAIL) 4.2

 

[Urriak Uruk]

No. There's an entire industry built around messing around with reviews, for reasons that aren't always obvious. (I don't think WotC is putting fake reviews into the system.)

This site suspects that 45% of the reviews on Hoard of the Dragon Queen are suspect, possibly to intentionally drive down reviews in a programmatic way.

The same issue happens whenever you let the public add reviews, most famously at Rotten Tomatoes.

When I followed your link, I got zero %... I think we may be in different countries?

 

[Mistwell]

When I followed your link, I got zero %... I think we may be in different countries?

They all start at or close to 0% until you request a full report. I just ran the report for today and it's now "31% of potentially unnatural reviews removed."

 

[Whizbang Dustyboots]

They all start at or close to 0% until you request a full report. I just ran the report for today and it's now "31% of potentially unnatural reviews removed."

Nice. Mine said they had scheduled another look, which I guess you got. Thirty-one percent is much better than 45%, but that's still a lot of problematic reviews.