I've got a perspective I've been mulling as I've been lurking this thread.
Exploration is essentially, information management as game play. At the most basic level, its about gaining information on the game space-- 'Whats on the other side of the door?' or 'Whats in that uncovered hex?' or 'Whats the king really after?' and finding out the answer is the reward for successful exploration. Some people do just enjoy working out the answers to questions, especially if they feel like it was a secret.
That reward becomes more meaningful if it can be leveraged for further rewards (such as using information about the layout of a dungeon to avoid nasty monsters, or using information from the king's journal to escape the palace via a secret passage, or learning a particular magic item is in a particular place.) This is the classic joy of seeing a locked door, finding a key, and using it to open the door, to get some reward out of it.
A puzzle is exploration when solving it involves gathering and utilizing information, a
Zelda Puzzle Box Dungeon is a well known example. Social Interaction can be (and often is) exploration as well, if you're asking questions and trying to get answers. Even combat can probably be exploration if you're using combat mechanics to gather information somehow (picking a fight to expose an individual you expect is hiding something?), although usually combat is a resolution (finding the thing you need to kill being the exploration 'question') or an obstacle ('we'd love to search this room, but there's an actual elephant in it we need to deal with.')
All of these 'classic' exploration mechanics we're discussing exist to spice up exploration by making it harder to perform. Torches being time limited for instance, or wandering monsters, adds pressure that creates natural questions 'how long do we spend searching this room?' with consequences you can weigh. They aren't 'exploration' per say, but they work with the existing exploration game play of 'pose question' 'discover answer' to spice things up in the same way a monster having a cool ability spices up a fight.
The reason exploration is often seemingly ephemeral, is that we're very loose with information-- if the players have a preset goal they don't have any questions to answer about what their goal should be, if the players know where they need to go and what they need to do when they get there then they don't have any questions to answer about how to complete their objective. We often see designers add side stuff as optional to reintroduce exploration when this occurs, where if you don't pose the question and try to answer it, you'll miss out on say, loot.
So a lot of this is down to how you structure your games, an adventure with an explicit plot and goal can easily skip the information management and make itself entirely about execution thereby eliminating exploration. A game where you have a goal, but aren't sure how to achieve it can have a bit more exploration in working that out. While, a game where you have neither an explicit goal, nor know how to achieve it, has maximal exploration as your goals AND how to achieve them as subject to your ability to pose and answer questions (and then of course, be decisive about what to actually do.)
At the end of the day, the question and answer loop IS exploration, and can be summarized along with its rewards as information management.
