Min-maxing is a subset of optimization, IMO. For most purposes, they can be used interchangeably. In general, they're a problem when they interfere with "normal" game play in any number of ways that can be summed up as differing expectations between participants.
For me, that primarily means anyone who spends more time on making sure the numbers work out than that the character is narratively interesting. The two aren't always in conflict, though -- I've played with a lot of folks who are just plain good at math and understanding the implications of certain choices, but still build the character personality first. At my table, it's a problem when one or two players are substantively more or less engaged in the numbers than the others, to the point that it's difficult or impossible to build challenges that don't either wipe out part of the group or force part of the group to either dominate or fade into the background.
At a macro level, I find it a problem when certain players and/or tables are substantively more or less engaged in the math such that things like "level appropriate" and Challenge Rating lose meaning across the game, as a whole. It's fine when a group recognizes they're well outside the "norm" and can express their style accordingly (giving new players a heads-up and/or interpreting the rules with a bit of salt). When they start to get into "badwrongfun" territory, it's a problem. It seems, to me, that this is more a problem with the optimizers because they are noisy. The more casual/roleplay folks tend to just walk away and/or are better at just ignoring the rules.