I think swapping primary attributes for various classes based upon flavor of subclass is a great idea, and I'm actually planning on it for the cleric and wizard in my next campaign. As an example:
Abjuration - WIS
Conjuration - CHA
Divination - INT
Enchantment - CHA
Evocation - INT
Illusion - INT
Necromancy - WIS
Transmutation - WIS
Likewise for the cleric domains:
Arcana - INT
Forge - INT
Grave - WIS
Knowledge - INT
Light - WIS
Life - WIS
Nature - WIS
Tempest - CHA
Trickery - CHA
War - CHA
You could also do it easily for the Bard as well if you wanted:
Glamour - CHA
Lore - INT
Swords - WIS
Valor - CHA
Whispers - INT
Now obviously while some are obvious selections of subclass to ability (Arcana and Knowledge clerics using INT, Enchantment wizard using CHA) others are less cut and dried. But I want to keep numbers relatively balanced across each class so that no matter what mental stat you roll high on you have some options for subclasses in each class, and most of them I can make the case why I went with one over the other (like for example the War cleric is the most paladin-like of its subclasses and thus CHA as its stat.)
None of these house rules are at all necessary, but its just a nice way to vary things up. Plus, it helps avoid the oddball situations that often seem to crop up, which are things like how most Clerics all seem to be really good at Perceiving things (because Perception uses WIS). This won't float most people's boats, but if it does, then I say go for it.
Abjuration - WIS
Conjuration - CHA
Divination - INT
Enchantment - CHA
Evocation - INT
Illusion - INT
Necromancy - WIS
Transmutation - WIS
Likewise for the cleric domains:
Arcana - INT
Forge - INT
Grave - WIS
Knowledge - INT
Light - WIS
Life - WIS
Nature - WIS
Tempest - CHA
Trickery - CHA
War - CHA
You could also do it easily for the Bard as well if you wanted:
Glamour - CHA
Lore - INT
Swords - WIS
Valor - CHA
Whispers - INT
Now obviously while some are obvious selections of subclass to ability (Arcana and Knowledge clerics using INT, Enchantment wizard using CHA) others are less cut and dried. But I want to keep numbers relatively balanced across each class so that no matter what mental stat you roll high on you have some options for subclasses in each class, and most of them I can make the case why I went with one over the other (like for example the War cleric is the most paladin-like of its subclasses and thus CHA as its stat.)
None of these house rules are at all necessary, but its just a nice way to vary things up. Plus, it helps avoid the oddball situations that often seem to crop up, which are things like how most Clerics all seem to be really good at Perceiving things (because Perception uses WIS). This won't float most people's boats, but if it does, then I say go for it.