Yes, I already got a thread closed due to this concept, and this is partly my attempt to exonerate myself. I didn't even need to use charm person as an example in the original thread, as the DC would be the same if the guy used dancing lights to impress his dates and get a +2 on a Charisma check. But because I did use charm person, everyone got really disgusted.
Using dancing lights is an indirect attempt to influence someone's emotions, like inviting them to a fancy restaurant, flirting, or what not, and aren't bad or even "gray area" territory. That's the case even though dancing lights is magic.
Charm person (or drugs, or Dominate, or what have you) are direct attempts to change someone's emotions (for whatever reason) and I think that's where the divide comes. A lot of people might see Charm as indirect, at least when it comes to getting someone to do something they wouldn't normally do.
So, leaving out any form of mating behavior, I do not think that the victim of the charm should automatically hate the charmer, or even realize they were charmed, provided there is no evidence to the contrary.
Note that someone can be charmed (in real life) to do things they wouldn't normally do, and might kick themselves afterwards, asking "why did I do that?"
However, it's been my experience that Charm gets used like Suggestion or a mild Dominate in a game, often as a way to advance the plot (eg get someone to talk, or get an unwilling king to allow you to break some laws, etc). Perhaps people have better examples of how Charm can be used, in ways that wouldn't give the victim a chance to realize what has happened. Also note that Charm has some fairly obvious components (IIRC) that the caster would need to hide so the victim doesn't realize they've been toyed with magically.