One simple solution: max of 5 level spellcasting PrC. If you have a hlaf spell progression, you'll end up 2 or 3 levels behind in caster levels. Which would still allow you 9th level spells. Second, you'll gain much less ability. These abilities will have to be worth approx familiar levels + 1 bonus feat (what a wizard would lose). The pre req will make it costly to the sorceror or the cleric, balancing it in regards to the wizard.
Even if the pre requ of the Red Wizard PrC are not much of a sacrifice, the loss of more schools is a good balancing idea IMO. Especiallly since another of the pre req is that you be a spec. wizard. I think the bonus feat is too much, but the rest is not that bad IMO. Better saves and better DC for one school vs loss of more schools, familiar abililties and feats.