You wouldn't have to take away low level slots, just stop giveing new ones, say once you can cast spells two levels higher. So when a Wizard hits 5th they get no new level 1 spell slots.
That's pretty much the way things are in 3e - wizards only get one more 1st level slot at 7th. Stopping the progression at 3 slots would only cost wizards about 10% of their slots at 10th or 20% at 20th. They'd still have a huge number of slots at high levels.