I'd agree it shouldn't but it's a matter of psychology. If you want to encourage behavior, you reward the behavior. If you want to discourage a behavior, you punish it.
Rewarding a behaviour you want to encourage is fine only until the reward becomes automatic, and thus expected.
Ideally, an exploration action could have one or more of three outcomes: reward (treasure, info, shortcut), punishment (trap, monster, misdirection), or nothing. Finding nothing is not a punishment.
A party is more likely to just ignore a door they aren't supposed to open.
You've had different players than I, then. Here, I could put every discouragement known to DMs on a door and they'd still try to find a way through it. (then again, how do the players know whether they're 'suppoesed' to open a door or not, until after they've already done it?)
And if they ignore it, what was the point of putting it there in the first place? It'll also confuse the players, because they'll think your descriptions are less interactive and more narrative. There's a door there, but is really a door or is it another stage-set?
In part you answered your first question with your second sentence: in part perhaps it is there to confuse, misdirect, or delay. As for the second question, the only way to find out is to go through it and look.
As the party, busy exploring deep in an old castle, opens a door and looks in I lay out the scene: "
Behind the door is a round room about 30' across. An open-grid metal spiral staircase leads upward from the middle and passes through a hole in the ceiling. An open archway across from you seems to exit directly into another room, on your left and right are closed wooden doors. A bright torch in a sconce above each exit provides light. The only furniture is three sets of shelves, curved to match the room's walls, standing one between each pair of doors except to your immediate left. On the shelves are an assortment of knick-knacks, a few books and papers, and - as you can see even from here - quite a lot of dust."
So you've got four known exits other than the door you're coming in: ahead, left, right, and up. What comes next?
Nobody really likes to look at nothing, though. It's boring and it wastes time on describing the nothing rather than actually playing the game.
Exploring and describing the 'nothing' is every bit as much a part of playing the game as talking with the King or putting a sword through a Giant. In the description I give above, how do you know whether it's 'nothing' until you interact with it? How do you know if any of the exits go anywhere useful until you check 'em all out?
Gaining something for searching a room isn't necessarily giving them a participation trophy because they won't know they didn't truly earn it.
Yeah, we're coming from completely different backgrounds here I think: I can't imagine a party
not searching a room unless I describe it as four bare walls, a bare floor and a ceiling - and even that'd make 'em suspicious!
I structure my dungeons fairly similarly. Despite my constant rewards, players tend to miss 1-2 treasure hoards in a dungeon because they either forgot to inspect something or they didn't think about it.
By dread, I don't mean fear. It's perfectly fine if a player hears your description of a crimson dragon opening it's maw and preparing to unleash a shower of flames. That's awesome. It's not awesome when players hear you say roll initiative for the group of mooks and the players audibly groan and bang their head on the table while a player turns on the shredder preparing to recycle the leftover of their characters.
What always both amazes and pleases me is that what on paper are a complete bunch of mooks can give a party fits yet that same party can turn around and knock off foes that in theory have them completely outgunned. I like this, in that I prefer a flatter power curve where low-grade monsters remain a threat further into the game, and at the same time low-level parties can take on major foes with at least a non-zero chance of success.
Most attacks from a balanced fight are designed to hit based on the bounded accuracy. AC doesn't get all that high even at high levels while to-hit goes into the +11's to +13's. It's rare to come up against an enemy whose AC drops your hit percent below 55%. Spells almost always do some form of damage as a guarantee. Even if the enemy succeeds against the save, the caster will get to roll their damage dice.
I'm not running 5e so I've yet to deal with the joys of bounded accuracy. That said, I like a mix of fights and opponents - some where the PCs can't really miss and others where they have a very hard time hitting, with most being in the middle somewhere.
Of course, exploration needn't be low risk-low reward. Traps up the ante for collecting treasure. It makes it very tense when the party finds an obviously booby-trapped cache where they must find the mechanism, disarm it, then grab the treasure without activating the trap. And it won't feel like they merely participated since they did have to make hard decisions and perform difficult tasks to acquire what they earned, and with a risk of damage or worse.
Agreed. However, one has to be careful not to overdo this.
And again, mixing it up is fun too. Sometimes I'll leave amazing treasure just lying there unguarded by anything*. Other times I'll have traps and monsters guarding nothing, or nothing of value. The minute they start to be able to predict what comes next, I know I'm doing it wrong.
* - in an adventure I wrote and ran a few years back there was a secret passage - just a shortcut between two otherwise-very-explorable areas, which they found and explored no problem - and lying on the floor of this passage was a Ring of Three Wishes, completely undefended by anything other than the passage being hard to find. The idea was that if they used this passage some PC might have a Bilbo moment and find the ring...so of course they never found the passage at all.