Individual behavior is anecdote, not data. Individual actual play is not a meaningful testbed or metric. Only in aggregate over many groups do we get anything reliable.
Your contention is that one home game is not meaningful, that is reasonable
However, my experience isn’t limited to a single group or a narrow context. Over the past four decades, I’ve run sandbox campaigns across home groups, virtual tables, store games, and convention events. I’ve coordinated and run live-action events and managed a LARP chapter where players joined and left constantly. True, my experiences doesn't come from evaluating survey spreadsheets, but far more extensive than one gained from a single home campaign.
I have done thing that are adjacent to methods like surveys, I’ve run my published sandbox Scourge of the Demon Wolf fifteen times with entirely different groups, all using my Living World techniques. Deceits of the Russet has seen seven iterations so far. Those aren’t isolated anecdotes, they’re stress tests. Sandbox adventures are notoriously hard to write about but by running so many session I can spot patterns of choices that give me a framework on which I can write an adventure useful to others.
But they also given me data on how different types of players react, adapt, and learn within my framework.
Umbran said:
Lots of GMs hide setting logic from players... so we can’t really assume players are aware of the probabilities coming out of setting logic.
That’s true in some games, but in mine, how I present things during the campaign is designed to teach players how the world behaves. My background in boffer LARPs taught me a lot about how people learn from incomplete information. LARP players don’t get a rulebook, they get a character sheet and an in-world situation. Yet they adapt, succeed, and often become highly proficient. As an event director, I watched this process unfold over years, and I applied the lessons directly to tabletop.
It’s about creating the right level of situational awareness. Through consistent roleplay, environmental cues, and verbal descriptions, players can infer risk, read context, and make informed decisions, without needing probability charts in front of them.
That’s not unique to my game, it’s a learnable, transferable skill in any immersive, responsive setting.
Umbran said:
I’m not asking you to meet a standard. You don’t need my validation.
I know you didn't ask me, but you did set a standard in your opening: that only broad aggregate evidence is valid.
I don’t need validation, but I do seek useful critique. That means weighing opinions from people with different assumptions, and not dismissing working procedures as anecdotal.