CapnZapp
Legend
The sad truth is that games like WoW has made us get used to truly well-tuned combat encounters.Yes, I think the basic idea is sound. Which is one reason I so badly want good tools. Ideally, you'd be able to post a challenge scenario somewhere (a number of pre-set scenarios, which people tackle either linearly or in some more complex flow), and people would play through the encounters (with actions/HP/etc. being tracked through the scenario, and little animations to show what happens in what order). Then they hit the Save button and it goes out to the cloud somewhere, and the scenario designer marks it as accepted or not accepted based on whether it conforms to the rules and expectations he had, and if it's accepted it goes on some kind of a leaderboard/database of encounter logs.
I think it would really be quite valuable to see how dozens or hundreds of DMs and players approach the same scenario, to get statistics like average number of HP expended, spells cast, risk of death, etc. That would help you see whether or not "Hard" and "Deadly" encounters are really as hard as they claim to be, and to analyze patterns in what makes them hard or not.
But the tools are not there yet to do this effectively.
The tool you envision would truly allow the devs to sift through hundreds of thousands of iterations of how a particular fight goes, and thus measure a truly objective "CR" of a particular monster.
But what for?
If most players using the tool are "soft" gamers not into charop, the CRs will be higher than what other, hardened, players would like.
Heck, if most DMs were doing their best to use the monsters at their disposal to fight back against the heroes, it would lead to the same thing.
(Not coincidentally one reason I haven't participated in this thread)
Basically, D&D relies on a human DM to smooth over CR wonkiness. This can on occasion catch an inexperienced or even an experienced DM with his pants down.
That is all.
Myself, I'd much rather WotC worked on a Monster Manual 2, to pad out the selection of higher-CR foes.
Then I can use these prepackaged high challenge foes while Flamestrike continues to challenge his players with the relatively meh MM1 foes.
Since I'm not using XP or CR guidelines, at least I don't have to worry about xp inflation.