Yes, the rules serve the DM, not the other way around. Which is why I say the limit is not the rules at all, but what the players are willing to tolerate. I, for example, wouldn't tolerate the DM encroaching upon the one thing I'm expected to do in the game - describe what my character does...