Could not agree with this assessment more! So long as each player in the group (including the GM) is roughly on the same page with regard to optimization, the game is perfectly viable for the whole range of builds, from hapless PCs whose players don't understand the basic math of the system to high-level PCs whose players have optimized every build choice for combat (or important single skill) effectiveness.
And, as you note, it takes real work and a combination of multiple build components to reach very high optimization (or a complete disregard for character mechanical effectiveness to be unplayable).