I just googled "most charismatic people in history" and the first entry gave me:
- Napoleon Bonaparte
- Fidel Castro
- Winston Churchill
- Mahatma Gandhi
- David Koresh
- Adolf Hitler
- Martin Luther King Jr.
- Malcolm X
- Nelson Mandela
- Teddy Roosevelt
- Eva Perón
- Aung San Suu Kyi
I went further down the page - tons of lists, citing hundreds of different people. Few of them particularly attractive by Hollywood standards. Quite a few of them notably the opposite. It seems that for most people, charisma =/= beauty.
So the claim that beauty is a part of charisma is harmful, not well supported, and totally illogical in a universe in which most sentient beings belong to a plethora of distinct species. Yet, if you still want your character to beautiful as part of your charisma, the rules let you do that. So I'm not seeing what the issue is.
Edit: for good measure, here are various dictionary definitions:
"the magnetic and charming personal quality that draws people to someone"
"a special personal quality or power of an individual making him or her capable of influencing or inspiring large numbers of people"
"the ability to attract the attention and admiration of others, and to be seen as a leader"
"a personal magic of leadership arousing special popular loyalty or enthusiasm for a public figure"
"a special quality conferring extraordinary powers of leadership and the ability to inspire veneration"