My point in this matter is not to be contentious or to imply that anyone who plays differently isn't playing in an immersive way. Instead, I am saying that the more immersive you play, the less meta-gaming will affect how the players choose to play their characters.
If players think in terms of stat blocks, it's a problem, in my eyes. They shouldn't know an enemies capabilities beyond what they've actually seen. So I strive to avoid that.
I'm sure you don't mean to be too contentious, because you're one of the most reasonable posters on the boards!
Nevertheless, what I've quoted is contentious. For instance, here is my conception of immersion (not just theoretical, but based on practical experience): the player feels an emotion that, broadly, correponds to the emotion experienced by the PC. So, if the PC is frustrated than so is the player; if the PC is feeling threatened then so is the player; if the PC is exhausted, then so is the player. I find this is more immersive than (say) pretending to be scared when you're not.
And I find that stats and mechanics are crucial to this. For instance, when my players were preparing (in character) to confront Orcus, they thought he might be tough. When I told them his stats (in response to the invoker/wizard player's successful monster knowledge check), they were genuinely worried.
Or, to give an example of frustration: in my 4e game the polearm fighter and archer ranger were affected by a chained cambion's Mind Shackles attack. Here is the flavour text of the chained cambion, and the mechanic of that ability (from the 4e MM3):
A chained cambion's reigning emotion is hate. It hates its life, its captors, and its enemies who roam free. A chained cambion screams its despair within the minds of nearby foes.
Mind Shackles (psychic) Recharge when first bloodied
Effect: Two enemies adjacent to each other in a close burst 5 are psychically shackled (save ends; each enemy makes a separate saving throw against this effect). While psychically shackled, an enemy takes 10 psychic damage at the start and the end of its turn if it isn't adjacent to the other creature that was affected by this power.
Aftereffect: The effect persists, and the damage decreases to 5 (save ends).
The archer and fighter had to stay adjacent or take damage. Which caused frustration (and required some clever acrobatics at some points, due to the terrain of the room). When one saved but the other didn't, and so insisted on staying adjacent, this caused the frustration to turn into resentment. The players really were experiencing the despair and hate screamed into their PCs' minds by the chained cambion. (This monster is one of my favourite ever bits of RPG mechanical design.)
As far as exhaustion is concerned, this should be the result of active player resources (not hit points, but things like Action Surge) - so the player gets to choose when to try (just like the PC doing his/her utmost), and then when the resources are all gone but the challenge is still there, the player (like the PC) experiences the feeling of having nothing left.
(In 5e, getting Inspiration should also be a part of this cycle - being able to go on.)
One of the ways I have achieved a more immersive game is to create encounters that are not meant to be resolved through combat....ones where if it does come to a fight, the PCs will lose.
I prefer these sorts of choices to be made by the players rather than the GM.
Here's some old posts that show what I mean:
The scenario I ran yesterday (from the Eden Odyssesy d20 book called "Wonders Out of Time") called for a Large bear.
I wasn't sure exactly how many 10th level PCs would be facing it at once, and so in prepping I placed a single elite level 13 dire bear, rather than a lower level solo bear (a level 7 or 8 solo would be a rough XP equivalent), because I thought the slightly swingier high level elite would produce a more interesting range of outcomes across a wider range of possible PC party size.
<snip>
As it turns out, the whole party encountered the bear. I didn't want to do any re-statting on the fly, so stuck with the level 13 elite. They players decided that their PCs would try to tame and befriend the bear instead of fighting it. To keep the XP and pacing about the same as I'd planned, I decided to run this as a level 13 complexity 2 skill challenge (6 successes before 3 failures).
in a "fiction-first" system, the players could attempt to avoid a combat because that offered their best chance of success. If you design the challenge of avoiding said combat "To keep the XP and pacing about the same as I'd planned", then you undo the value of that choice.
I strongly disagree. Wide variance in difficulty or rewards based on player strategy doesn't preserve the value and meaning of player choice, it destroys that value - essentially, you create a single correct choice.
<snip>
if a diplomatic approach is just as hard as a fight, whether or not the PCs have good CHA, skill trainings, etc means something. The fact that the characters chose a non violent means of resolving the problem even if it wasn't any easier tells us something about their values. If talking is easy, then PCs can get through without strong social skills, and all that their choice tells us about the characters is that they're expedient.
When one choice is obviously superior, going for it is a pretty trivial decision.
I agree with Victim. The less there difference to prospects of success the choice of tactics/approach makes, the more it expresses the values of the players (either outright, or in character as their PCs). Conversely, the more that the prospects of success are responsive to choices of tactics/approach, the more that those choices reflect the means/end rationality (ie the expedience) of the players.
An additional dimension in the latter case is whether the GM has set it up on purpose, or not. If the GM has done so, then the players making the "right" choice also shows their ability to respond to and follow GM cues.
And a final thought, that links this part of my post with the previous (about mechanics): if the players have reasonably deep pools of resources, then you can combine the two approaches. That is, players can express their values by choosing to expend resources to make their preferred approach viable. For instance, in
this session, the player of the fighter PC spent resources to succeed on an Intimidate check and thereby confirm, in his first interaction with a Primoridal and its followers (Yan-C-Bin and some djinni), his status as the divine jailer of those elemental creatures who oppose the gods and thereby threaten the world.
And in the same session the players spent resources to persuade Maruts - who were present to observe the beginning of the end times - that they had got the date wrong, and the end times weren't yet coming. This wasn't necessarily easier than fighting the Maruts (the PCs are quite good fighters, and their diplomacy efforts meant that one of their number had to spend a couple of turn solo-ing the Tarrasque), but it was important to the PCs (and hence to their players) to establish it as true that the end times have not yet come.
TL;DR: My approach to establishing an immersive game is to ensure that the mechanics enduce a state in the players that corresponds to their PCs; and to frame the ingame situation so that the players can rationally choose to engage it in a way that expresses their values and desires, rather than purely expediently.
True free form campaing came with the Shadow Dale, Tantras and Waterdeep adventures that were introducing 2nd edition.
Some of use were running true free-form campaigns well before this, eg in my case using Keep on the Borderlands, and the D-series, or making up my own stuff.
Yet for all its good side and innovation, 4ed fell short in players and DM expectations. The classes were too balanced at every level and playing one or the other didn't have any meaning at all. The game almost looked like a table top MMORPG.
I think it's generally wise to avoid false generalisation. 4e didn't fall short in my expectations - it exceeded them. And it didn't look anything like an MMO (I've not played them, but I've seen them played, and heard reports of the play of them, and they don't seem to resemble anything like what I've described from my 4e games in this post).