D&D General Data from a million DnDBeyond character sheets?

[FrogReaver]

What does the character ID signify? Could a player (or DM) use the same ID for different characters, the digital equivalent of erasing and reusing a character sheet? Or could a DM use a single ID for every character in a game so as to avoid the players having to pay for subscriptions? Etc.?

It appear to be either a character ID assigned by Beyond or possibly the aggregator of the data. The ones that are duplicates are all the same exact character with the same character id or the same character and character id at a different level or with a different feat or different stats. For our purposes it appears to be a unique identifier for a given character.

Character IDs range from 1 to about 1.5 million.

 

[FrogReaver]

I just went through reimporting my data. Had much fewer import errors after converting it to excel first. Only 8 vs 30,000 when doing so from csv. Either way, even if somehow that meant i don't have 8 characters in the data I'm not worried about that impacting the results. Chances are those rows weren't actually data rows in the import and thus the unimportable values.

After reimporting my next step was to remove duplicates. After doing so I have 404,320 records, with 404,315 unique character ids. The 5 are duplicate character ids and either have different levels or different stats, different feats, etc. I plan to leave them in as the 5 entries shouldn't really change things and determining which to remove seems more problematic given the small number of entries.

I guess the next step is to determine the logic for triming out invalid characters, like ones with 0 in a stat or don't have a subclass and should based on level. Etc.

 

[FrogReaver]

Starting 404,320 Records

Removing any character with a 0 in base hp or a stat i end up with 327,198 records
Additionally Removing any character not level 1-20 = 327,195 records
Additionally Removing characters with stats outside the 1-20 range = 327,121 records
Additionally Removing characters with negative hp = 327,041 records
Additionally Removing characters with name Like "test" = 324,463 records
Additionally Removing characters with name Like "NPC" = 324,309 records
Additionally Removing characters with all 8's for stats = 322,397 records

 

[ichabod]

What does the character ID signify?

It's the character ID on D&D Beyond. It's the number you see in the URL when you are viewing the character.

 

[ichabod]

Removing any character with a 0 in base hp or a stat i end up with 327,198 records

I would also recommend removing characters with all 8's for abilities. This is what happens when you select point buy but then don't spend any points.

 

[FrogReaver]

I would also recommend removing characters with all 8's for abilities. This is what happens when you select point buy but then don't spend any points.

I like that one. It's these next 3 you had listed i'm not so sure about.

• Have a background - There's 52,000 characters that look otherwise complete and have no background. That's alot to exclude.
• Have a standard class (PHB + Artificer) - The main non-official class i see is Blood Hunter. Unless others have issues I think I'll keep all classes
• Have an official sub-class - Is there an easily accessible list of all official subclasses out there? If so I'll see what this looks like.

 

[Ferrousbones]

Additionally Removing characters with stats outside the 1-20 range = 327,121 records

I don't recommend this because it is possible for some normal characters to reach as high as 24.

• Have a background
• Have a standard class (PHB + Artificer) The main non-official class i see is Blood Hunter. Unless others have issues I think I'll keep all classes
• Have an official sub-class Is there an easily accessible list of all official subclasses out there? If so I'll see what this looks like.

If it is a character intended for play, it will have a background, race, and class, plus a subclass if level 3+; so if there is no background, then not a valid character.
For good data, they need to be classes from or endorsed by WOTC, but since custom classes aren't really possible with Beyond, this should be fine.
Well, there is a site classified as piracy. Also, does Beyond at least list all the options, even if they cannot be selected? I know it did with non-core races for a while at least.

 

[FrogReaver]

I don't recommend this because it is possible for some normal characters to reach as high as 24.

If it is a character intended for play, it will have a background, race, and class, plus a subclass if level 3+; so if there is no background, then not a valid character.
For good data, they need to be classes from or endorsed by WOTC, but since custom classes aren't really possible with Beyond, this should be fine.
Well, there is a site classified as piracy. Also, does Beyond at least list all the options, even if they cannot be selected? I know it did with non-core races for a while at least.

Good call on Barbarians being able to reach 24. I'll raise that to 24.

I've played many games without backgrounds or by choosing 2 additional skills only. I don't know how common that is but i'd be hesitant to assume ~15% of the characters simply didn't finish selecting a background especially given my experiences. That's a ton of class, race and other data to arbitrarily throw out just because the background is missing.

I know little about how Beyond functions.

 

[Oofta]

Good call on Barbarians being able to reach 24. I'll raise that to 24.

I've played many games without backgrounds or by choosing 2 additional skills only. I don't know how common that is but i'd be hesitant to assume ~15% of the characters simply didn't finish selecting a background especially given my experiences. That's a ton of class, race and other data to arbitrarily throw out just because the background is missing.

I know little about how Beyond functions.

It doesn't force you to put in a background.

 

[FrogReaver]

It doesn't force you to put in a background.

Does it force the other selections? Race, Class, Stats, etc?