For me, this is one of the greatest things about RPGs - the ability to step away from the limitations of rules enumerating what the PCs can do and having them do stuff anyway because it makes sense.
Indeed, I think that's the primary advantage of D&D over computer games -- the ability to do the unexpected.
What's best in D&D to me?
1) The ability to do things the designer didn't expect you to do.
That can be full sandbox play, event driven or dungeon-crawling play where you don't do as the adventure author expected, or railroads where you go off the rails.
It can be CaW-style combat -- pulling out the alchemist fire or smoke sticks, using cover, using improved initiative sucker punching the bad guy before he knows the fight started, dropping food to distract the predator, using Unseen Servant to do something unexpected, etc.
Or a little broader like finding local allies to help you a little in the fight ("Rio Bravo" adventuring rather than "High Noon" adventuring -- not you alone against the baddies, but you and people you're helping against the baddies, even if they can't help much.)
Or much broader, like when my party was SUPPOSED to deal with the rebel baron by sneaking in the castle and killing him, but instead built a protest movement, studied the law and used the legal right to the redress of grievances to get an audience with him, and used a good argument (and a Diplomacy roll of natch 20!) to convince him to change his position instead of fighting him at all.
2) Combat that FEELS dangerous.
People don't really have to die, but it should feel like they could, at any moment. Which means swingy and unpredictable and a little scary, not grindy or balanced. You have critical hits, you have relatively low HP relative to damage, and maybe you even have some save or die effects.
And you have ways to stave off danger at a cost, like healing magic that's limited, potions that are rare, and perhaps -10 hp before you actually die (instead of just being nearly dead).
If a PC is going to die, harebrained rules lawyering interpretations to prevent it are OK. For example, once a PC druid was knocked to -1 hp by a shadow in a fight in a graveyard, next to a temple. They were supposed to be dead and turn into a shadow. I ruled the PC would turn at the end of his next action, so the others had a chance to save him. The cleric was too far to reach him and heal him. But another PC pulled a "Speak With Animals" to tell the druid's pet dog -- which acted on the same initiative as the druid -- to drag the druid into the temple, which he guessed (correctly) would be under clerical magic that prevents the undead from forming. So the druid was just unconscious, not a shadow.
Really exciting rolls by the enemy -- like a make or break attack when it's a near TPK -- should be rolled in the open, not behind the screen, so everyone can gasp or cheer at once.
3) The world makes sense.
Goblins need food and water. Goblin armies have logistics you can attack.
Humans have politics, laws, social classes, jobs, and economy. Trade makes sense, based on comparative advantage and logistical costs (so limited overland trade in bulky, low valued commodities). It's a working "Magical Mystical Society".
It's generally a low-level, gritty world. No magic street lamps or singing teapots, unless something special is going on. No magic to replace technology.
Most people use a plow and an ox to eak a living from the soil. The local tavernkeeper might have adventuring levels (especially if she's a retired PC from another campaign), but is more likely to be a low-level Commoner or Expert with appropriate skills to running a tavern.
Your local village doesn't have a magic store. It might have a hedgewizard (Adept) and probably has a Cleric of 1st-3rd level.
If the city even has a magic store, it has a limited inventory (because supply and cash to buy it are rare), you can't rob it without bringing down the rage of whoever put up the capital for it and the mages guild that gave it permission to operate. You can't even get discounts, because the guild regulates the prices. And like Tony say, the rules support the world making sense. Magic dealers must pay less when buying than when selling (regulated guild rate is 80% in my campaign), and there must be a profit margin for people who make magic items, to use a detailed example.
4) Zero to hero.
PC's start as gifted folks, but not much better than others, and follow the same rules as NPC's. PC's have a background -- and skills related to the background -- for what they did before they became adventurers.
PC's can advance quickly, and become important or famous. People who can cast 3rd level spells -- miracles like Fireball, Fly, and Create Food and Water -- are rare and important, and may become well know.
At higher levels -- maybe 7th-9th -- PC's are majorly important, and may get strongholds, followers, etc. They are truly "who's who" at this point. The game can change from dungeon crawling to building a settlement.
5) Story matters, the characters are important to the story's outcome, and their actions change the world.
There's more going on than just the "killing things and taking their stuff". With great power comes great responsibility, and the PC's must help "the good guys" or the good guys are in real trouble.
The PC's significantly drive how the story ends, and their actions will have a "persistent" effect on the world, which will change future situations for them and for other parties in the same world.
Because a PC party back 25 years ago now defeated the G123Q1 threat, Geoff and Sterich never fell, and my Greyhawk setting is "rewritten" to exclude that part of Living Greyhawk, for all my Greyhawk campaigns.
Actions can have unexpected, sometimes bad, consequences, too. A party in 1998 snatched Daoud's Wondrous Lanthorn from the Lost Caverns of Tsojcanth for Bissel, eluding the Kettite patrols and adventurers who also wanted it. Shortly afterwards, Ket invaded Bissel, a major change in my version of Greyhawk, and the Lanthorn was a factor in that.
6) PC's are persistent.
Retired PC's, or PC's of someone who stops playing (or in one case, dies in real life) don't just disappear. They become NPC's in the background, sometimes important to the story of patrons/friends/helpers to the current PC's.
Superstar NPC's who were PC's in Gygax's campaign -- like Robilar or Mordenkainen -- may be mentioned in the background or (once) could even appear, but they never give PC's orders or use them as flunkies, as such folks were accused of doing in 2nd Edition FR modules. They aren't fundamentally different from the retired PC's or other NPC's -- they have their goals, and they are mortals.
7) All NPC's and monsters have their own goals, and act to achieve their goals to the best of their ability and knowledge. NPC's/monsters should be ROLEPLAYED, not played to defeat the PC's or be beaten by the PC's or "make a fun fight".
NPC's/monsters shouldn't do things to "make the fight interesting", but because they think it's what they think they should do. They can't act on knowledge that the DM has, but the NPC doesn't. When appropriate, some NPC's should fight to kill like tactical genius rat-you-know-whats, others should make rookie mistakes like opening themselves up to an AOO (if they are rookies or stupid). Monsters fleeing for their lives or surrendering should be definite possiblities.
That's my manifesto for now. I think it's mostly a CaW/grognard manifesto, but it's probably somewhat different from what other people think is CaW.
