• Welcome to this new upgrade of the site. We are now on a totally different software platform. Many things will be different, and bugs are expected. Certain areas (like downloads and reviews) will take longer to import. As always, please use the Meta Forum for site queries or bug reports. Note that we (the mods and admins) are also learning the new software.
  • The RSS feed for the news page has changed. Use this link. The old one displays the forums, not the news.

Orr Group's Roll20 Stats Q2 2019 Use New Method; Cthulhu Up, D&D/PF Down

Roll20 has posted its latest usage stats for Quarter 2, 2019, using a new method. Instead of only using "user selection data", they are now looking at the usage of character sheets across their 4 million accounts. As always, D&D comes in first, and is followed by Pathfinder and Call of Cthulhu (which rank differently based on whether you are looking at players or campaigns). The new counting method has resulted in some changes!


tumblr_bcab2a189617f303e183d717d34d36bd_3574c11e_500.jpg


tumblr_5351d79be5c6ffa8f5f83fbe7cf4496c_29cb711e_500.jpg


Screenshot 2019-07-26 at 11.13.45.png



There's also a lot of uncategorised games -- 13%-14% of the total, which includes custom character sheets with no listed system, or games without character sheets.

One interesting facet of the Call of Cthulhu rise is that the average game length is about a quarter of that of other systems -- "We found that they have an average playtime of 11.9 hours, compared to 40 hour average game times for other systems on these charts."

So how's D&D doing? Back in 2015, D&D had 25% of games and 31% of players. In early 2018 it has surged to 61% of games and 70% of players. This year, however, the figure has dropped to 51% of games and 54% of players. It's hard to tell whether this is just due to the new method of counting.

Looking at those same three reports, Pathfinder had 21% of games and 40% of players (fairly similar to D&D's stats) in 2015, in 2018 had dropped to 10% of games and 27% of players, and is now sitting at about 6.5% of games and 8% of players.

Call of Cthulhu, the third placed game, began with 1.3% of games and 8% of players in 2015, increased to had 2.6% of games and 7.3% of players in 2018, and has grown in the last year to 9.5% of games but dropped to 3.4% of players.

As I mentioned, it's hard to tell how the new counting method has affected these figures.


tumblr_1972ae64e703551a5c9ff1181f974839_d813754d_1280.jpg
 
Russ Morrissey

Comments

darjr

I crit!
I wonder if they can post what the old counting method numbers are. We could get a better idea then.
 

Nylanfs

Registered User
This is a much better method of counting games rather than what people put on their profile. Although it doesn't give total number of games. Also does this count when GM's are prepping for their games, or only when players are connecting to their sheets?
 

Morrus

Well, that was fun
Staff member
It’s the same method I think D&D Beyond uses. Though that suffers from large numbers of practice builds rather than those used in play.
 

volanin

Explorer
Something is not right with these pizza charts. For example, the Campaign Percentage for D&D is 51.87%, but the red band in the chart is waaaaay over 50%... more like 60% or 65%.
 

kenmarable

Explorer
The discrepancy between the pie chart and the numbers is because that Top 10 only accounts for 88.69% and the pie chart is based on just the Top 10. So D&D 5e in the Campaign Percentage is 51.87% overall, but is 58.48% of the Top 10.

If you dump those numbers into a spreadsheet, and build the chart based on just the Top 10, it looks precisely like the one shown.

So, it might have made more sense if they included an 11.31% slice for "Everything else."

Edit to Add: Also, for the Account Percentage data, that Top 10 only covers 86.94%. That's why D&D 5e's 54.44% is shown as 62.62% on the pie chart.
 

kenmarable

Explorer
Also, I was curious and did a little digging into what they meant by "user selection data" as well as wondering whether percentage drops in Pathfinder were overall lowering in the number of games & players, or whether there was just larger growth in D&D 5e, for example.

It is not a different counting method, but an entirely different data collection method. (AKA They aren't parsing the data differently, they are looking at entirely different data.) Previously, it was just relatively small sample sizes based on if a user set "Games I enjoy playing" and the "Games I'm looking for" fields in their profile, as well as if Campaigns set the game they are playing field. All of which are not required, and are primarily used for Roll20's "Looking for Games" system.

(I know I never set those fields. Considering the relatively small sample sizes in previous reports, it looks like the vast majority didn't either.)

Their current method of looking at what all users are actually using (especially if they can account for inactive games) is FAR better than what they used before, but also, unfortunately, comparing this report to previous ones is maybe not apples & oranges, but at least apples & pears. ;)

So the previous data was heavily skewed towards the subset of players & DMs that are looking for games on Roll20, and largely excluded those who set up a Roll20 game with people they already knew. So some (or most? or none?? or all???) of the shift between previous reports and this one could also be explained by differences between those data collection methods looking at different populations.

Call of Cthulhu could have grown in popularity. Or maybe Call of Cthulhu players tends to already know each other and aren't looking for games with strangers as often in Roll20's Looking For Games system. Could be the opposite with Pathfinder - maybe they are more likely to be "Looking for Games" than other systems.

Also, it would be nice if they included the actual data sizes. Previously they had sample size counts (but those were just self-selected subset of users, so wasn't real useful), but now it's just "4 million+ accounts." So it's impossible to tell, for example, whether Pathfinder actually declined or possibly grew but just at a slower rate than others (or changed due to the difference in data collection methods mentioned above). So even if the percentage dropped, the actual number of players & games might be lower, higher, or the same. There's no way to tell.
 

dave2008

Adventurer
[MENTION=1]Morrus[/MENTION], please correct me if I am wrong, but since they are using a completely different method of counting user data, then can we really ascertain that D&D and PF have dropped. It should really be an independent chart shouldn't it?
 

Morrus

Well, that was fun
Staff member
[MENTION=1]Morrus[/MENTION], please correct me if I am wrong, but since they are using a completely different method of counting user data, then can we really ascertain that D&D and PF have dropped. It should really be an independent chart shouldn't it?
I thought that was what I said.
 

timbannock

Explorer
What's still interesting to me is that AD&D predominantly rises above any other OSR by a pretty big factor, but so much of the OSR tries to emulate B/X or OD&D. OSRIC, the AD&D clone, has a highly active playing and forum community yet seems to be ignored in terms of getting an SRD out, having a working website and easy means to get books, or get many compatible releases on DriveThru. People just straight up say "Compatible with 1e and BX" more often than choosing a specific clone any more.
 

DMMike

Game Masticator
Looks to me like D&D is followed by Indie Games, not Pathfinder. Personal bias, maybe? Although that Systems list looks pretty thorough...
 

volanin

Explorer
The discrepancy between the pie chart and the numbers is because that Top 10 only accounts for 88.69% and the pie chart is based on just the Top 10. So D&D 5e in the Campaign Percentage is 51.87% overall, but is 58.48% of the Top 10.

If you dump those numbers into a spreadsheet, and build the chart based on just the Top 10, it looks precisely like the one shown.

So, it might have made more sense if they included an 11.31% slice for "Everything else."

Edit to Add: Also, for the Account Percentage data, that Top 10 only covers 86.94%. That's why D&D 5e's 54.44% is shown as 62.62% on the pie chart.
Thanks for that!
I assumed "Uncategorized" was "Everything Else".
 
Last edited by a moderator:

Mistwell

Adventurer
Since uncategorized means character sheet that isn't the Roll20 supplied one, or no character sheet, I suspect the largest portion of those are 5e and PF2, since both have character sheet generators out there in high quantity. For example, if your players are using DnDBeyond to print character sheets and using Roll20 with those sheets in their hands, it's getting set as Uncharacterized rather than 5e. Same probably goes for PF.
 

dave2008

Adventurer
The title states that Cthulhu is Up and D&D/PF are down. However, we can't truly know that if the numbers are using different data from 2015-2018 than they used for 2019.

This would be kike saying my son grew 6" from 2015-2018 but he shrank in 2019 because he weighed 160 lbs. Different data, we can't compare them.

We really have no idea how 2019 compares to 2015-2018.

However, this is all moot if the graph presented is based on the same data, i.e. they went back and re-figured the 2015-2018 figures with the same date/method they used in 2019. Do you have any idea if that is the case?
 
Last edited by a moderator:

LordEntrails

Explorer
Am I just being pessimistic and assuming they are not actually providing any raw number because they don't want to show a decrease in active users, games played, number of campaigns or anything else that could be compared to the growth numbers of their competitors?

I agree this is a much better way to collect data, but just telling us percentages really keeps the data from being very useful for comparison to anything else , including the older reports.
 

kenmarable

Explorer
Am I just being pessimistic and assuming they are not actually providing any raw number because they don't want to show a decrease in active users, games played, number of campaigns or anything else that could be compared to the growth numbers of their competitors?

I agree this is a much better way to collect data, but just telling us percentages really keeps the data from being very useful for comparison to anything else , including the older reports.
I agree that it is less useful without knowing the sample size (other than imprecise “4 million+”). However, to offer a more charitable motivation, it could just be that when their sample sizes were a small fraction of their users, that is important to be upfront about. Now that their sample size is all users, they might have thought it wasn’t as important. After all, this isn’t a separate research firm, this is just from the programmers who build Roll20. Statistical research and polling isn’t their specialty, writing code is, of course.

Alternatively, you may be right that they are intentionally hiding their exact user numbers. Even if they aren’t going down, a lot of businesses (across many industries) seem to think revealing that sort of info helps competitors. Heck, even with D&D being wildly successful, WotC is pretty secretive about actual sales numbers. So not publicly stating that info isn’t necessarily a sign of losing business.
 
The two releases (Roll20 and Fantasy Grounds) are marketing releases. They show increasing user counts and that a lot of the most popular game is played on their system and plenty more. Roll20 has always marketed their user account number because it is a big number and they never discuss how many are active and playing because that is not a positive story for marketing.

The stats are interesting and I am glad that both companies release them even if they are marketing exercises.
 

Advertisement

Advertisement

Top