Not who you asked, but I'll say that my experience matches yours.
The first problem with spell points is that spell levels aren't linear, but spell point costs almost always are. Really, you want 1st-level spells to cost ~3, 2nd-level spells to cost ~5, and 3rd-level spells to cost ~12. But 4th level spells aren't really that much better than 3rd level spells. 4th level spells really are just another set of 3rd level spells for the most part. As you progress on, the spells are increasingly explicitly designed to be limited by how many you can ever cast in a day. And then you get to 6th level spells and they're supposed to be really potent as the original highest spell level... but they aren't? 5th and 6th level spells feel markedly better than 3rd and 4th level spells, but they're basically equal to each other. Then you get to 7th-level and higher and the list is just almost entirely crazy stuff. Sure, some of it is just modified fireballs, but the good stuff that really draws people in really should be magic items or quest rewards or unique rituals.
Really, I think to make spell points work you need to throw out spell level entirely. Each spell should have it's own costs, maybe even with it's own means to limit recasts. At that point, you should probably throw out the whole magic system entirely and bring in something completely different. Maybe something skill-based, or dice-pool based.