Orr Group's Roll20 Stats Q2 2019 Use New Method; Cthulhu Up, D&D/PF Down

Roll20 has posted its latest usage stats for Quarter 2, 2019, using a new method. Instead of only using "user selection data", they are now looking at the usage of character sheets across their 4 million accounts. As always, D&D comes in first, and is followed by Pathfinder and Call of Cthulhu (which rank differently based on whether you are looking at players or campaigns). The new counting method has resulted in some changes!

​

There's also a lot of uncategorised games -- 13%-14% of the total, which includes custom character sheets with no listed system, or games without character sheets.

One interesting facet of the Call of Cthulhu rise is that the average game length is about a quarter of that of other systems -- "We found that they have an average playtime of 11.9 hours, compared to 40 hour average game times for other systems on these charts."

So how's D&D doing? Back in 2015, D&D had 25% of games and 31% of players. In early 2018 it has surged to 61% of games and 70% of players. This year, however, the figure has dropped to 51% of games and 54% of players. It's hard to tell whether this is just due to the new method of counting.

Looking at those same three reports, Pathfinder had 21% of games and 40% of players (fairly similar to D&D's stats) in 2015, in 2018 had dropped to 10% of games and 27% of players, and is now sitting at about 6.5% of games and 8% of players.

Call of Cthulhu, the third placed game, began with 1.3% of games and 8% of players in 2015, increased to had 2.6% of games and 7.3% of players in 2018, and has grown in the last year to 9.5% of games but dropped to 3.4% of players.

As I mentioned, it's hard to tell how the new counting method has affected these figures.

 

[darjr]

I wonder if they can post what the old counting method numbers are. We could get a better idea then.

 

[Nylanfs]

This is a much better method of counting games rather than what people put on their profile. Although it doesn't give total number of games. Also does this count when GM's are prepping for their games, or only when players are connecting to their sheets?

 

[Morrus]

It’s the same method I think D&D Beyond uses. Though that suffers from large numbers of practice builds rather than those used in play.

 

[volanin]

Something is not right with these pizza charts. For example, the Campaign Percentage for D&D is 51.87%, but the red band in the chart is waaaaay over 50%... more like 60% or 65%.

 

[darjr]

Maybe the chart is using the old method?

 

[kenmarable]

The discrepancy between the pie chart and the numbers is because that Top 10 only accounts for 88.69% and the pie chart is based on just the Top 10. So D&D 5e in the Campaign Percentage is 51.87% overall, but is 58.48% of the Top 10.

If you dump those numbers into a spreadsheet, and build the chart based on just the Top 10, it looks precisely like the one shown.

So, it might have made more sense if they included an 11.31% slice for "Everything else."

Edit to Add: Also, for the Account Percentage data, that Top 10 only covers 86.94%. That's why D&D 5e's 54.44% is shown as 62.62% on the pie chart.

 

[kenmarable]

Also, I was curious and did a little digging into what they meant by "user selection data" as well as wondering whether percentage drops in Pathfinder were overall lowering in the number of games & players, or whether there was just larger growth in D&D 5e, for example.

It is not a different counting method, but an entirely different data collection method. (AKA They aren't parsing the data differently, they are looking at entirely different data.) Previously, it was just relatively small sample sizes based on if a user set "Games I enjoy playing" and the "Games I'm looking for" fields in their profile, as well as if Campaigns set the game they are playing field. All of which are not required, and are primarily used for Roll20's "Looking for Games" system.

(I know I never set those fields. Considering the relatively small sample sizes in previous reports, it looks like the vast majority didn't either.)

Their current method of looking at what all users are actually using (especially if they can account for inactive games) is FAR better than what they used before, but also, unfortunately, comparing this report to previous ones is maybe not apples & oranges, but at least apples & pears.

So the previous data was heavily skewed towards the subset of players & DMs that are looking for games on Roll20, and largely excluded those who set up a Roll20 game with people they already knew. So some (or most? or none?? or all???) of the shift between previous reports and this one could also be explained by differences between those data collection methods looking at different populations.

Call of Cthulhu could have grown in popularity. Or maybe Call of Cthulhu players tends to already know each other and aren't looking for games with strangers as often in Roll20's Looking For Games system. Could be the opposite with Pathfinder - maybe they are more likely to be "Looking for Games" than other systems.

Also, it would be nice if they included the actual data sizes. Previously they had sample size counts (but those were just self-selected subset of users, so wasn't real useful), but now it's just "4 million+ accounts." So it's impossible to tell, for example, whether Pathfinder actually declined or possibly grew but just at a slower rate than others (or changed due to the difference in data collection methods mentioned above). So even if the percentage dropped, the actual number of players & games might be lower, higher, or the same. There's no way to tell.

 

[dave2008]

[MENTION=1]Morrus[/MENTION], please correct me if I am wrong, but since they are using a completely different method of counting user data, then can we really ascertain that D&D and PF have dropped. It should really be an independent chart shouldn't it?

 

[Morrus]

[MENTION=1]Morrus[/MENTION], please correct me if I am wrong, but since they are using a completely different method of counting user data, then can we really ascertain that D&D and PF have dropped. It should really be an independent chart shouldn't it?

I thought that was what I said.