The biggest "exploit" in being able to choose your casting stat would be in multiclassing. For example, if a sorcerer can use wisdom, then they can dip a level of monk and improve their AC significantly. Effectively, it makes it much easier to build a SAD multiclass character.
If I implemented...