It's possible for the game rules to set the random table. That's what classic D&D does. And there are rules for adjusting the result relative to the dungeon level. So within the limits of the randomness of the roll, and the possibility of a trap or trick taking the PC to an unintended level, it is the player who gets to choose between challenge and cakewalk, by choosing dungeon level.If a DM is being a neutral adjudicator, a curious explorer of the fiction and playing the world with integrity then it's not going to be long before the players push play toward something that he at best can provide a high level detail about. But players will typically then ask about lower level details - which forces the DM to have to fill such details in. One example might be, you've encountered a small band of orcs (DM can fairly make this adjudication based on the players geography, their stealthiness, etc). Questions will arise: How many orcs, how far away, are orcs typically hostile, can I identify what group these orcs belong to, etc.
At this point the DM really only has one course of action - make up the number of orcs and other relevant details, being as faithful as he can to the integrity of the world. Sometimes a world might have a specific answer, but most won't (at least not for all such questions). In this instance it's my opinion he can be both neutral and actively pick a challenging number as neither the integrity of the world nor his neutral adjudication is going to be compromised by having 8 orcs (a challenge) instead of 4 (a cakewalk). I do think there's a limit on how much number of enemies can be chosen.
I think it's also important to note that randomness alone cannot resolve this problem as the DM would also be responsible for setting the random table of results
But if the GM is making that call, then I don't see how they are also being neutral.
I understand your point here.When you start making those decisions guided by what would make the best story or what would challenge players in the middle of the session the skill of navigating the fictional environment basically means nothing. If GM is free to stop being a referee at any moment they deem necessary that's exactly what causes that sense of Mother May I? because there is no telling on what basis from moment to moment decisions are going to be made.
In practical terms, what tools - other than an impossible degree of prep - do you think can solve the "how many Orcs" problem? In my Torchbearer adventure design on the weekend I used the adventure creation procedure to make a general decision about the number of obstacles of various degrees of difficulties, and built my creature numbers within that framework.
Maybe the adventure design guidelines count as an example of this? I'm trusting that the designers have got their numbers right to make the game work, just like a classic D&D GM needs to trust that Gygax or Moldvay has got their "number encountered" spread basically right.If the GM is going to act with an agenda for play than we need some other way to maintain competitive integrity because the GM is not doing so, not even attempting to do so.