DM fiat, at least part of the time, works when the DM is consistent. And when a game isn't all fiat, but well-targeted fiat, you can really get something that sings, feels "real", but doesn't get bogged down in too much detail. So how do you get that?
If the DM is a jerk and intends to stay that way, you don't. But then you have a bigger problem than consistency anyway.
If the DM tries hard to be consistent, and he starts with something like 1st ed. AD&D--then the DM puts a lot of work into it. He reads gaming materials and historical data to extrapolate from. He thinks about it. He puts in processes to flag inconsistencies and correct them over time. He takes notes. After he has done this for several adventures, he gets his system down and gets gradually more consistent from there on. This is how a lot of us learned it back in the day.
Yeah, yeah - I did that apprenticeship from around 1977 to 1999 or so. I went through all the phases - the "biggest, baddest dungeon dude", the "my world is a jewel" phase, the "story is everyrthing" era - the whole nine yards.
Eventually, though, I have come to the belief that GM fiat (note both parts of that - both are important) is hardly ever a good way to frame a set of rules. Before writing this off, consider:
Let's define the game "Rules" (capital 'R') as representing the "physics" of the game reality - not necessarily in the way real world physics do, but in the sense that they describe how the game world actually works. A system that invokes "GM fiat" is not a set of Rules, per se. It's a statement about where the Rules dwell - they dwell in the mind of the GM. How the GM visualises the game world represents the actual game world "physics" - the
actual Rules of the game.
The players, meanwhile, are supposed to play characters who have (generally) lived in the game "reality" all their lives. That they should be ignorant of how the world works seems extraordinary in the extreme. That they should have to guess about the real efficacy of their own powers and abilities seems bizarre.
I propose, therefore, that the players, as well as the GM, should always have "inside knowledge" of the game Rules. I don't think we need to be prescriptive about how that "insight" is ensured though. Two possibilities spring to mind - both are evident in existing roleplaying systems:
1) The Rules of the world are defined by the rules of the game; the game system describes reasonably well how characters interact with the world in explicit terms. D&D 4E and GURPS are both examples where the majority of game situations
can be played out without resort to DM/GM fiat or other arbitrary Rules creation during play.
2) The rules of the world are generated via collaboration and, if necessary, systemic resolution between all of the players - including the GM, if there is one. PrimeTime Adventures and Universalis are games I have played where this is the mechanism used to decide in-game outcomes based on the aesthetic/dramatic sense and tastes of the full gaming group.
Either of these methods can work fine; the first is better if some oppositional tension between players is desired (e.g. "gamist" D&D) or if mystery and investigation is to be a component in play (Call of Cthulhu, for example). The second is better if the desire is to build a world based on aesthetic and "artistic" values - worlds trying to be "realistic", for example, or trying to generate "Story Now" via Narrativist play focus.
For me, though, at least, the place for a game which relies on the aesthetic taste
of one person only has narrowed to almost nothing. Games for teaching children how roleplaying works, maybe - provided they are not carried on too long, teaching "bad habits".
Such games do seem to remain curiously popular, however - mainly, I phant'sy, with GMs... But maybe I'm being overly harsh.