It's almost like, when you have actual rules that both sides genuinely stick to and only break with group consent, they help engender mutual trust and respect When both sides feel they are getting a fair shot and can know that they should make use of the tools available to them to the best of their ability, players are more likely to feel, and become, engaged, and GMs are more likely to play rough and get excited about what they can potentially do.
When you take all the rules away and it's all "whatever That One Guy says, goes," suddenly you need to have that trust just to get off the ground, and anything that endangers that trust imperils the entire exercise. Nobody knows where they stand anymore, so every motive becomes questionable and every deviation becomes suspicious, whether on the GM's side or the players'.
Or, to put it in uselessly pithy terms: It's almost like "DM Empowerment" was kind of a bad idea to begin with.