There are two things that
Charm Person is good for.
The first is "random townspeople charming." Charm the hostile merchant to get a better deal. Charm the young lady at the local pub. Charm the king so he won't want to sic his guards on you. In this case, you can usually be subtle enough that no ill-will is detected from the spell. You leave the scene before it ends, and all that happens is that they think you were kind of keen for some reason they can't explain. In this capacity, it is an
excellent narrative spell for your character, and really expresses the archetype of the enchanting arcanist who gets in your head and has that allure of the unknown. This can help you gain access to certain areas that are prohibited otherwise, in most of my campaigns. If the guards see you as a friend, they'll trust you inside, at least for a moment.
The other is "clever battle avoidance." A random group of insular dwarves in a dungeon might shoot first and ask questions later, but if you can just talk to them first (and use Charm Person), you might convince them to stop fighting you and maybe even team up with you. It won't be very useful after you start killing 'em, but if you can persuade the party to take a different tactic on these critters, it might prove a better outcome.
The key in both situations is that either the target doesn't really find out about the mind control (they just think you're persuasive, and you're there and gone fast enough so that they don't really register it), or that, once they find out, they trust you anyway (having won their trust through more mundane means as well). They might still suspect something (spellcasters are known for having the power to alter your minds), but it doesn't really matter to them because you've either proven yourself, or are out of the picture entirely.
It's a magical leg-up on the winning friends and influencing people game, and it's a useful shortcut for when words would take too long, or to establish a quick-and-dirty trust baseline (albeit one you need to quickly enhance with more mundane methods).
Basically, I want a charm person that simulates charm, not mind control. Buying you a beer, not a roofie in your drink. After someone gets done turning on the charm in real life, you might feel slightly used, but you might feel kind of flattered that they gave you all that attention. You might even be friendlier afterwards.
I want a charm person you could use to cheer up an angry baby. And I don't see in the spell description why you shouldn't. Do I have to make it a house rule?
I think all of these are pretty acceptable, while under the influence of the spell. Charm Person should cheer up an angry baby, and it should be able to get you a free drink from some goon at a bar. If you tell someone under the influence of the spell that they've just been magically charmed, feeling flattered would be a hilariously plausible result. And you absolutely shouldn't be poisoned by someone who is your "friend" under the influence of the spell. Even Evil people should recognize the value of friendship and comradeship, or at least "temporary alliances."
The thing about the spell is that it doesn't last all that long, in the end. It doesn't make you their friend for more than a few minutes. If you want to be their friend afterwards, you either need to charm them again, or use more mundane means to support yourself. And most people, as this thread has pointed out, aren't that stoked about having their minds altered. If you're still their friend when the magic wears off, they might forgive you ("I wouldn't know how cool a guy he is if he didn't!").
And, as a side note, IMC, people who have been charmed don't necessarily know they've been charmed. A spell only has its explicit visual effects. Even if they save against it, they only know that they saved against some mental assault. Only a trained spellcaster (or someone who made an Arcana check) would be able to say "You tried to charm me, didn't you?!"