I don't know... it seems a weird rule
Isn't it very much like your rule? Aren't you the one that equates having and using out of character knowledge with cheating? Are you going to go on to present very similar ideas as to what constitutes cheating in this very post?
what do you do to enforce any rule...
In the rare case that my authority is questioned I just say, "I'm the GM. Ultimately I decide what the rules are." Most players recognize I have authority over the game by setting down to play with me as the GM. But notice there is a difference between having authority over the game and having authority over out of the game, which is what you are asserting.
I disagree... we can all just do our best to compartmentalize (again not saying anyone is perfect at it)
I would have agreed with you 30 years ago, but I have tried to do this way too often and long ago realized it was impossible. I will again give a concrete example.
We are in a swamp and we are attacked. I the player recognize we are attacked by a troll. Now, I've never given any thought before whether my character would be able to recognize a troll or know anything about them. The other players don't yet recognize it is a troll because they are new to the game and they are trying to attack it with normal weapons, which I know as a player is going to be useless. So now, I'm tasked with trying to compartmentalize and figure out whether or not my player would recognize a troll. So I pretend not to know and also attack the troll, knowing it's going to be (mostly) useless. But, at the same time, I also know that in not attacking the troll with fire, I am still metagaming. Because I can't know whether or not my character would recognize a troll. I'm probably only doing what am I doing because I want to not "cheat", not because of what my character may or may not know. I also can't know how fast my character would figure out that weapons don't work and we need to burn a troll in absence of knowing that fireworks on trolls. So I'm thinking to myself, "How many rounds should I wait before it is fair to figure out that I should try a different approach?" and "Should I be making intelligence checks for my character to see if my character would figure this out fairly?" And I'm doing all this because I want to be the player that I would want if I was the DM, and know "good players don't metagame". And at that moment I realize that my whole approach to the sin of "metagaming" to this point has been entirely wrong. I realize there should be no pressure on the player to not use their out of character knowledge because it ends up creating silly, unfun, and probably unrealistic situations where the player is being asked to play against themselves. The stance of a being player doesn't support not using out of game knowledge in the general case.
And so I dropped that whole load of crap for what it was and never looked back.
my way seems way more open functinal and fun.
And I'm telling you, I've played your way and my way and at least for me, it very much isn't.
the solution is to in your mind run a mini simulation and think not "How should I the player react in the best way" but instead "what would the character do with only what they know"
That's not remotely a solution. I've tried it. I tried for years, and the more conscious I became of the process, the more I realized it was a lode of hooey and impossible. You can't know what the character would do if they didn't know what you do. There is no unbiased simulation of that process, and if there was it would take your choice as a player out of the equation so that the character would 'play itself' absent your control over it and you'd be reduced to watching the simulation instead of playing the game.
I don't understand... do you not even try to think about what your character knows, how they interact with the world?
Of course I do. But that doesn't solve the particular problem we are talking about here. That might solve, "Even though I am not, my character is an Anarchist and his principles would compel him to act in this way in this situation." That doesn't solve the problem of, "I know what a troll is." that I described above.
I don't brag about my IQ, and I am neither the oldest, the smartest nor the longest playing member of my friends...
Neither am I. The point of the statement is that it's not easy to Sherlock Holmes me and say, "What would Celebrim do here?" and that if you make choices based on that rather than in game facts, don't be surprised to find you guessed completely wrong. I have funny stories about that, but they'd be a distraction at this point.
if it was out of character why would they WANT to metagame it...
That's what I'm trying to tell you. It's not a matter of want. If a character has out of game information they can't help but metagame in some fashion. There is no unbiased simulation. There is no process that takes that knowledge out of your head.
this is a great example. a few years ago (precovid) we had an NPC traveling with us. The DM did not hide that he was the big bad Out Of Game... all of us Out of Game knew it. in game we trusted him and treated him as a friend and brought him everywhere... he even sat in as we planned the assault on HIS castle.
That's a great example? That's the way you play. Gosh not only does that sound boring, OMG is that railroading in metagame director stance. "This is the BBEG but I want you to treat him like a friend because you don't know that." End of session, I send an email, "Thanks for the good time, but I'm really busy right now and don't think I will have time to attend any more."
now there may have been a bit of surprise fun taken away...
Yeah, like I'm comparing that to the like 3 year long reveal of who Tarkus the Necromancer was in my last D&D campaign, and yeah... I know which situation my players would prefer.
as a DM I ask if they really wanted to play the game, and offer an alt game. I also start a discussion on how we as a table feel about it.
as a player I do the same (theory PC 3) and say "Hey we should be a little less on this" and ask for an out of game discussion
So your response is to stop the game and enforce the no metagaming table rule. OK then.
yes, one seems to be in character the other out... I would think nothing (DM or Player) about this. I would most likely (PC or NPC) agree.
the second one seems like fun to me the first one seems boaring and weird.
The first and second are the exact same situation, only the player of PC #1 in the second situation has decided to try to hide his motives so that you don't get angry, stop the game, and it's ruined for everyone.