What is good for D&D as a game vs. what is good for the company that makes it

[wedgeski]

I feel like you're dancing around the question. How can you determine anything when you don't know the motives, reasons or whatever for why a particular character was built in the CB? Taking your above example... you have no idea what percent of vampire characters are seeing actual play. All you know is 10,000 have been built with the CB...what exactly does that tell you?

With respect I'm not dancing around anything. You're saying a CB character must see play to be a useful data point, I'm saying I don't agree. Good trend analysis can play all sorts tunes on data like that.

To your particular point... sure you do. A character seeing regular use will have a quite different profile to one that doesn't: consider frequency of change, frequency of levelling up, frequency of being printed, just as a few examples.

 

[Imaro]

With respect I'm not dancing around anything. You're saying a CB character must see play to be a useful data point, I'm saying I don't agree. Good trend analysis can play all sorts tunes on data like that.

To your particular point... sure you do. A character seeing regular use will have a quite different profile to one that doesn't: consider frequency of change, frequency of levelling up, frequency of being printed, just as a few examples.

I'm not saying a character "must see play" to be a useful data point... I'm saying how do you correlate the data to a particular trend if you don't even know what the trend is. The only thing you know is x number of Y were created and a subset of the x number of y are or are not updated. Without knowing how or why they are being updated... how does that data help you (without making some pretty big assumptions) as far as improving your product or sales?

As to your second paragraph... what is regular use? There are people who play every week, bi-weekly, monthly, quarterly even yearly. At what point do you know whether something is being used "regularly". I mean you can create a character and never play him but play around every so often on the CB to level him up... wouldn't it appear that he is being played regularly even if infrequently?

 

[Hussar]

Gaming still doesn't have that. We don't know that all the stuff being generated in the online tools is seeing actual play. A subscriber could log in and make up 8 characters in the builder. All we know is what data was used to build those characters not what those characters were used for.

It could be a player working on ideas trying to decide what to play. In that case only 1/8th of the data generated is seeing actual play.

That actually wouldn't be hard to track at all.

How many characters are accessed multiple times? Set your search parameters so that you only pick up characters that are loaded up, say, 5 times over a period of time and you have a pretty decent idea of what is being played.

While there are going to be a bajillion characters that get made, you only have to narrow the search a bit to find a more statistically solid bunch of information.

Heck, you could look at leveling patterns over time. A character that is leveled up, say, five separate times (say more than three days between leveling up) is most likely a character that is seeing play.

THAT'S the level of information that WOTC is getting from the DDI.

 

[Hussar]

I'm not saying a character "must see play" to be a useful data point... I'm saying how do you correlate the data to a particular trend if you don't even know what the trend is. The only thing you know is x number of Y were created and a subset of the x number of y are or are not updated. Without knowing how or why they are being updated... how does that data help you (without making some pretty big assumptions) as far as improving your product or sales?

As to your second paragraph... what is regular use? There are people who play every week, bi-weekly, monthly, quarterly even yearly. At what point do you know whether something is being used "regularly". I mean you can create a character and never play him but play around every so often on the CB to level him up... wouldn't it appear that he is being played regularly even if infrequently?

That sort of behavior would get smoothed over by the numbers though. What percentage of users actually do this? I'm pretty sure that someone with a decent background in statistics can separate that out pretty easily. While you might pick up the odd outlier, by and large, it wouldn't be that hard to find, nor would it mess with the numbers significantly.

All you have to do is do multiple searches, changing the parameters. How many characters are only accessed once a year? Twice a year? Twice a week? Once you have your base set, you can set parameters based on behavior.

While it's not an exact science, statistics never are. I'm pretty sure that with tens of thousands of users, you can make a pretty solid picture of what DDI players are playing.

 

[ExploderWizard]

While the data gained from CB use does tell something, that something may not be what WOTC really wants to know. Passive data collection via the Big Brothering of software use is too heavy on the assumptions.

If WOTC really wants to know what people are playing, then just ask them.

 

[prosfilaes]

While the data gained from CB use does tell something, that something may not be what WOTC really wants to know. Passive data collection via the Big Brothering of software use is too heavy on the assumptions.

If WOTC really wants to know what people are playing, then just ask them.

It's not cheap to interview 50,000 people, and there's a lot of patterns over time that could be pulled from CB that would take extensive repeated interviews to get. And surveys often have weird effects that give false information.

 

[Hussar]

While the data gained from CB use does tell something, that something may not be what WOTC really wants to know. Passive data collection via the Big Brothering of software use is too heavy on the assumptions.

If WOTC really wants to know what people are playing, then just ask them.

What assumptions?

Heck, you can data mine this all over the place. Track how often rules are accessed in the Compendium for example. If a particular rule is being brought up more often than others, it might be a sign that it could be simplified or explained better.

Trying to do surveys is incredibly expensive, and is every bit as subject to problems as this.

Then again, there's nothing stopping them from doing both.

 

[Imaro]

What assumptions?

Heck, you can data mine this all over the place. Track how often rules are accessed in the Compendium for example. If a particular rule is being brought up more often than others, it might be a sign that it could be simplified or explained better.

Trying to do surveys is incredibly expensive, and is every bit as subject to problems as this.

Then again, there's nothing stopping them from doing both.

Emphasis mine...


You really don't see the assumption you made here. It could also mean that the rule just comes up alot but may be too complicated to remember fully for many thus it's just easier to look up... or it could mean the rule is so obscure most don't remember it but when it does come up almost everyone has to look it up. See assumptions that could all be right or wrong.

 

[Hussar]

But that's the point Imaro. That rule gets flagged because it's being brought up more than others. Once the flag is raised, then you examine why that particular rule gets looked up more often.

However, those weren't the assumptions I was questioning. I was questioning Exploder Wizard's assertions that there are too many assumptions associated with data mining the DDI.

 

[Imaro]

But that's the point Imaro. That rule gets flagged because it's being brought up more than others. Once the flag is raised, then you examine why that particular rule gets looked up more often.

Okay, I get that... I guess what I'm saying is that before deciding courses of actions based on this data...I hope WotC does due diligence on the "why" as opposed to making their own assumptions as to the reasons.