It's important to remember that a dungeon is merely a convenient framing device, a tool for the DM to guide the flow of an adventure without resorting to blatant railroading. A series of boring slugfests does not magically become fun because you put it in a dungeon.
With that in mind, all the usual principles of encounter design apply. One thing I find tremendously helpful is to think, "Why do the players care about the outcome of this encounter?" If the only answer you can come up with is "Because they have to complete it in order to reach the next encounter," then it needs to be reworked or eliminated.
Beyond that, if you want the dungeon itself to be memorable--work it into the encounters. Include clues about the dungeon's background and purpose, and provide rewards for players who put the clues together correctly. That will encourage them to think about the dungeon as a real place rather than just background fluffery.
(However, beware of putting in massive lumps of exposition. You want to tease your players. Make them work to extract the history of the dungeon from you, one scrap at a time, rather than shoving it in their faces. If they decide not to bother, that's how it goes; figure out how to be more enticing next time.)
A coherent theme also does wonders for enhancing the feel of a dungeon. You can approach this in a few ways.
The simplest dungeon theme is the "creature theme," wherein the denizens consist of a specific creature type plus various hangers-on. For example, an "orc dungeon" would contain a lot of orcs, plus an assortment of critters that orcs might keep as pets, guards, or slaves. This is easiest to do, but can be somewhat limiting in terms of encounter variety.
More versatile is the "environment theme," in which the dungeon environment drives the concept. A "tomb dungeon" is an example of this; you could have undead rising from their sarcophagi, animated statues set to serve as guardians, and an assortment of vermin that have moved in over the centuries. The central feature of this dungeon is that it is the resting place of the dead, and all the encounters flow from that conceit.
Finally, you can have an "owner theme." This type of dungeon is ruled by a single mighty master (dragons and wizards are the most common). It's different from a "creature theme" because there's only one of the master's creature type--a dragon lair seldom has more than one dragon--but all of the dungeon's other denizens live in the master's shadow, and its presence is the single most important factor in their lives. Some may be scavengers feeding off the remains of its kills. Others are prisoners, or slaves, or guardians. Still others might be the original inhabitants who now skulk about in terror of the behemoth that has usurped their home.
