One big mistake a rules designer can make is thinking that he can solve issues of game mastery and adventure design with rules design alone. No rules set is so good that encounter design and adventure design are rendered purely mechanical issues. You can provide guidelines, but if you don't provide examples of play, then you haven't provided much. If you only provide mechanical examples, then you haven't provided anything. Another related mistake I see in some games is a system with great flavor and great mechanics, but where it is clear that the designer has stopped at that and never seriously thought about how his game looks like in play. Or if he has thought about it, it's a secret known only to him and which cannot be divined from the text. In that case, the designer should have spent less time making a 'tight' rules set and more time communicating the real mystery of his game, which is, how he himself runs it.