Systems I've had experience with:
D&D Psionics--The standard 3.5 psionics system is basically the 3.5 magic system altered to work more smoothly. If you enjoy the general power level of standard casters but abhor the inflexibility of fire-and-forget and spell slots, psionics is the system for you.
d20 Modern FX--This isn't quite an alternate magic system in its own right, as it's merely a short reprint of the 3.0 magic and psionics rules. Still, while the means to achieve the goal is largely the same, the goal itself is somewhat different--Because d20 Modern spells are harder to get, and because powerful non-magical weapons and devices exist in d20 Modern, Modern spellcasters are less about blasting foes and more about the interesting non-offensive abilities they can acchieve through magical means.
d20 Star Wars Force Powers--As a non-jedi character in a starwars game, I spent the entire time sitting on a sidelines while the force-using characters accomplished the party's goals. SW force powers by themselves might not be broken, but combined with the other things making jedi cool, it was hard to tell.
Green Ronin's Psychic Handbook--Similar to force powers, and a precursor to the system used in True 20. Unlike force powers, I've never seen problems with these (though they've seen only limited use in my modern campaign). They fit the feel of modern day "psychics" pretty well. I've heard these rules described as fairly low-powered compared to the standard D&D casters, but I've never actually seen both rules used in the same game.
Sleep Dep Creation's SFX Skills--work similarly to the Psychic Handbook rules, but more individualized for specific magical disciplines. So far, only Diabolism and Enochian Theurgy are out, and I've only used Diabolism. Very good flavor, but limited in scope and not really suitible for a standard high-fantasy game.
Elements of Magic: Mythic Earth--This is the one system I own but haven't used. It's skills based, and highly flexible, and it looks pretty awesome. It's too broad in scope and too powerful for the low FX modern campaign I run, but if you're running a high-magic game and want a flexible, skills based system similar to Ars Magica, then it looks like a great choice.