I rationalize the availability of magic much the same way as many previous posters, but here are some points I did not see in why I would allow a fair amount of magic to be up for sale.
The party is not the only ones in a campaign world capable of making magic items. This is something that has already been stated several times. IMO the vast majority of the "background" people would have plenty of time to make magic items for a living. Simply because they are not insane enough to "always" be adventuring. Plus, making magic items is pretty lucrative, and much safer than adventuring.
For churches, selling potions, scrolls, and other items is a great way to generate revenue maintaining their church, and maybe being able to afford to add on or build another church elsewhere someday.
So this is the general view i have of most camapign worlds, such as Greyahwk, Forgotten Realms, Scarred Lands, Oathbound, and Dungeonworld to name a few.
So the result on the game (at least ones I run) is that you can find potions or scrolls with relatively little effort, of minor power. Most magic items I decide on in a case by case basis, usually resulting in a percentage chance that a stock (in the DMG) item is available, until the items start being worth more than 15 to 20 thousand gold. Then I make it necessary to find someone capable of making the desired item and negotiate a contractr for making the item and then waiting the weeks or months for them to get around to, and actually make, the item contracted for.
Are there a lot of magic items in my campaigns? Yes, Does the party have a lot? Usually. Is it unbalancing? No, but it can be a major pain to keep it from becoming so. The main thing in my favor, as the DM, is that most offensive and some defensive magic items can only be used one at a time. So what if they carry around 4 wands, 2 rods, and 2 staves, 9 potions, 6 scrolls with 23 spells, a ring of djinni summoning, etc... They can only use one at a time.
Besides, when they really get encumbered with magic, they do occasionally fail saves against fireballs, etc... Or I can arrange for them to get caught in a no save is possible zone. Then I have them make saves for their magic items, and a "1" always fails, no matter how powerful the items enchantments or the caster level of the item maker. Mean? Yes. But it sure is effective at keeping the power level of the game under control. Which makes for a good long running campaign.
My favorite way to remove magic from a party is when they walk around town, alone. Rogues, anyone? Independent or guild affiliated will do. Just realize that the party may all of a sudden decide to declare war on the local Thieves Guild, even if one of their thieves didn't rob them. Guilt by association, you know.
I have even had wizards knock them unconcious/immobile with a lucky hold person or similiar spell. The cruelest one I ever did was when a character had a charm cast on him and he gave all his stuff to his "friend" who was in dire need (dire charm, anyone?) of said possessions to protect them and help them survive against these nasty enemies that were after her. She only needed them for a little while, and being that they were such very good friends, she would return them shortly. They never did find out who "he" was.
So a low level magic game is good, and is lots of fun. But a high level of magic is still very doable and fun too. Low magic games do have a lot fewer headaches and are easier to plan challenging encounters for, as well as are easier to maintain a game "balance", so the game doesn't "run away" from the DM's control, which is a must have for a successful and long-term campaign.