Okay, so, let's examine what parts of the feat make it out of balance.
a) Bluff (Int or Wis)
b) Diplomacy (Int or Wis)
c) Gather Information (Int)
d) Intimidate (Wis)
e) Charisma check using an Intelligence or Wisdom check instead, but at -1
If you're playing a character with a high Intelligence, odds are quite favorable that you're playing a wizard, though you might be playing a rogue. If you're playing a wizard, it doesn't seem that, qua wizard this feat does much for you (unless, perhaps, you're a wizard using charms or illusions or something, where the ability to persuade or manipulate others might come in handy). If you're playing a rogue, then, being able to Gather Information better than others, or having the ability to make better Charisma checks doesn't seem a big deal (honestly, how often does a +3 or +4 to these break a game?). Bluff, Diplomacy, and Intimidate, however, might get too much better, especially combined with certain other feats or abilities.
Does removing or weakening the feat by addressing those skills (Bluff, Diplomacy, Intimidate) weaken the feat too much to make it useful?
Maybe, indeed, the whole idea should just be shelved. There are lots of ways for a high Int or Wis character to compensate for a low Charisma.
Dave