Zhaleskra call for "elegant simplicity" reminds me that my own preferred way of thinking about complexity is not "simple vs complex," but "elegant vs complex" and "simple vs robust." That is; a simple system has ease of use at the expense of meaningful choices, while a robust system offers scads of meaningful choices but necessarily has a steeper learning curve, in order to understand all the possibilities the system offers. "Simple vs robust" is a design choice. "Elegant vs complex," on the other hand, should not be a choice, and elegance should always be preferred. Elegance is doing more with less, and complexity is doing less with more. A robust game will always be more complex than a simple game, but even a robust game should be as elegant as possible. Undue complexity is superfluous. The design goals of a game like Rifts are robust, but the game (IMO) is unduly complex. A game like Nexus has similar goals, but is a much more elegant game.
Paradoxically, you could have a "simple-complex" game, where there are few meaningful choices, and a lot of meaningless ones. Tables and tables of weapons that are fundamentally similar would be an example.
One way to make a robust system more elegant is to divide a big list of choices into two shorter lists. Consider races and classes in DnD. In older editions, you could choose a race or a class (that is, races were classes). When you could choose a race and a class, character creation became much more robust without much more complexity.
Thinking along these lines about weapons, consider a short list of weapon types that can have one of a short list of qualities applied to it. You can get the same degree of robustness as a long list of weapons, without the same degree of complexity.