I realize a lot of people don't like the fact that physical appearance is rolled into Charisma, but that doesn't mean the rules should be ignored...
At the end of the day, how you play your game is up to you. The rules are a guide, not a codified set of laws where the punishment for breaking them is death. Just look at the House Rules forum, and most of the d20 products that are available now. They all change the rules, or expand on them, in a way that the author thinks improves them. If you want to use Cha like that then, yay for you, if others think its bad, than they can do it their way.
I think at the end of the day it's up to YOU to decide whether NPCs in YOUR setting would be freaked out by a dude with huge scaly eyes and whether that constitutes a Cha penalty.
I've "ignored" the rules at least once in every gaming session I've ever played, simply because I've either forgotten the rule or have ruled differently based on things specific to my campaign setting. So sue me...
This is exactly why having a psudo seventh stat comliness is a goo idea
Charima should just be your force of personality and comlinees should be your looks
Theres to much variety in the races to lump looks and personality together
I do like the idea of seven stats, and I've seen rules for a 7th stat, but frankly, I can't be bothered to go through every skill and decide whether or not attractiveness affects it. At the end of the day, the only thing that might be affected by looks is how an NPC responds to a PC when they first meet, before they've spoken, after that the character's charisma takes over, when the two start a conversation. That's when the PC's charm kicks in.