I ran a campaign set in a world with destabilized magic, which made things more expensive and created a low(er) magic world. Here's some of the stuff I tried:
-have "well crafted" versions of items. This was mentioned earlier in the thread. I made 3 tiers of quality in my world, with smiths of great renown making really, really great mundane items. They gave bonuses comparable to magical items, but had to be made for a specific user, and were very difficult to obtain. You could make them as common as magic in D&D though, if you want to maintain power levels.
-increase the cost of magic. This makes it far less viable for people to have. I added a x4 multiplier on the generic bought magical items. Why hire 1 guy with a +1 sword when you could have 40 men with clubs? (For the record, they're all the same level, before anyone says +1 sword man would be more experienced.)
-make spells more difficult to cast, or more rare. If people haven't seen magic, then they're more impressed. As PCs, when they have little access to that kind of power, it becomes more special to them when they get it. Withholding it to a certain degree will likely make it more special when they do get it. I like the idea of spells taking longer to cast, but you could also increase the effects to make up for the delay. (Assuming you want to keep all the power levels the same.) So when that fireball that takes 3 rounds to cast goes off, it does 3x the damage. Blamo. Makes wizards more feared, but also gives them a hideous Achilles heel. Or more of one, I guess. Another way
-make magic controlled/hated Have a guild of mages, to which all mages have to belong, who jealously guard the secrets of magic. Again, due to its rarity and difficulty to obtain, then it becomes more special when it's acquired. Or, have the populace and ruling churches hate magic. Magic will then again be something terrifying, because no one has seen it, and there are rumours and stories surrounding it which make it horrifying. "Magic is powered by demons! All those who witness magic will see their children die slowly! Magic makes you sterile! Magic users will be sawn in half, then hung, then drowned, then burned!" You get the idea. Makes players more cautious when using it, the populace doesn't have it, but certain elements will still have it available. The sanity rules in Unearthed Arcana tie into this well. If all magic users eventually go insane, then you have a bunch of powerful, destructive psychos on your hands, something which regional governments wouldn't want... It's a good reason to make it controlled, and also makes players much more wary of its power.
-each magic item is unique Eliminate all the mundane magical items. Make players discover what the abilities of an item are over time, (essentially like an artifiact would be, though maybe not as powerful). They don't have to be superweapons either, a sword that you can command to glow in darkness, or which can be quickdrawn without a feat (leaps to the wielder's hand in danger, or on command, just for flavour) is still really cool, valuable and unique. Give them names, histories, stuff like that, and dole them out very slowly. Give the party a bunch, perhaps one intended for each character, and then as they discover their powers, have them distribute them amongst themselves as appropriate. Make up your own abilities for them. (What about a bow that weakens enemies to the next spell you cast, effectively raising the DC? A necklace that allows the wearer to perfectly mimic any voice? A bag of holding that has enough air to last days?) Do things the players don't expect, and force them to experiment with things, rather than flat out telling them, "It's a sword, you get +1 to hit." Cursed items are fun too, you can do lots of interesting stuff with that as well. (Gloves which instantly fling any weapon/item they touch at the ground? Terrible if you want to hold a sword, but if you can get your hands on your opponent's weapon, he's screwed too!)
...wow, that's gotten really long-winded. Hope there's something useful in there.