Your explanation makes sense. However, I would suggest providing magic item-like benefits to non-casters (if you are not already doing so), to balance out the magic items they would otherwise get. Otherwise, certain monsters and opponents become impossible for them, without the intervention of magic-using classes. Which relegates non-magic using classes to "second-class", or even more so. No pun intended, but I do happen to like puns. I am assuming, here, that you are not playing D&D 4th edition, in which case removing magic items would seem to provide no real downsides that cannot be fixed by adding perhaps +1 to-hit, defences and damage per 6 levels.