In which case, I've seen, CHA as a powerful stat can actually start to get out of hand. In a relatively low level game I was in, one character, a swashbuckler classed guy, if I remember correctly, had engineered his character to have a monster Diplomacy bonus (in 3.5.) Maxed out CHA, maxed out ranks, synergy bonuses, skill focus, etc. He was routinely hitting a 30 or more on Diplomacy checks--at a crazy low level, like 3rd or 4th or something like that. It got to the point where it was difficult to find reasonable reasons why he should fail to charm anyone at all that he decided to make the attempt on.
But yeah, I think that game was probably an outlier in terms of what the PCs did and who they interacted with. For the most part, using your CHA (unless you're a sorcerer or bard, or maybe a paladin) is just a once in a while perk, not really a game-changer.
That said, for my current group, we actually think it's quite amusing when we're in these situations and nobody has a high CHA. Or the guy with the high CHA is arrogant and aloof, and doesn't care. Or whatever. We're one of those kinds of groups that thinks terible things, hijinks, and character flaws sending the game to the crapper is actually highly amusing to us as players. We don't personally identify with our characters to the point where we internalize bad things happening to them as if they were bad things happening to us. As such, we're actually a fairly inept adventuring group most of the time, but we wouldn't have it any other way, because it's highly entertaining to have a group made of up folks more like Cudgel the "Clever" than a highly efficient strike force. So we enjoy having a group of unlikeable people who are socially somewhat retrobate, and who make our patrons and clients somewhat uncomfortable to spend much time with.