Most of the bad experiences I've had with "low magic" campaigns can be traced back to the same mistake: the DM assumes it's a simple change, and doesn't attempt to adjust all the interrelated aspects of the game.
"I just won't allow magic items to be bought."
(The party is nearly wiped out by a low-CR monster with DR, since they don't have magic weapons; only the offensive spellcasters can do much.)
"No one can take a spellcasting class, they're supposed to be rare."
(The party takes a month to heal up from a minor skirmish, and gets wiped out by the first guy they meet who knows Fireball)
And so on. D&D has been balanced to a rock-paper-scissors style of unstable equilibrium, where each class can beat certain types of enemies easily and lose to other types. Likewise, it assumes the classes have access to some items; if you don't have an Armor AC AND a Natural AC AND a Deflection AC and so on, you'll get hit way too much.
Removing one type of character/enemy without adjusting all the other related parts of the game is just asking for trouble. A good DM will have no problem with this, because he'll already be considering all the other things he'll have to adjust, but a bad DM won't realize that he needs to tweak DR and resists and regeneration and AC and... you get the idea.
Frankly, in my experience, the best "low magic" settings are those that take a nonmagical system (like D20Modern) and add a relatively open magical system to it, one that doesn't scale with level so well or doesn't give so many spells per day at high level, but that has more flexibility than the D&D slot system. Or, something like Four Colors To Fantasy, where you add a very flexible "Hero" class; even though practically all of the PCs will take some levels in it, they'll still take the other classes at some levels. In my opinion, D&D just isn't set up to be low-magic, or at least not without a LOT of work and the need to second-guess everything you bring in from normal D&D sources.