They don't really have an ironclad argument unless you never correct the issue. If the mistake that happened is just a mistake in game rules, you correct it later and then move on.
Correcting it later means one of two things has to happen:
1 - The erroneous action needs to be retconned to its correct version, which potentially invalidates every second of play that happened after it due to the one-thing-leads-to-another effect; or
2 - The erroneous action is allowed to stand in isolation, meaning that your rulings are now inconsistent with themselves (which I think we all agree is bad bad bad).
And so, getting it right the first time becomes important.
Some of us prefer not to have players who are antagonistic towards each other. If I was playing in a game and another player character kept stealing from mine and the DM was fine with it then I'd quit that game. I would rather a game where I play with others, not against them.
Note that characters being antagonistic to each other
does not mean the players are being similarly antagonistic. The players might be laughing their hearts out while their characters chop each other to bits.
If I'm playing in a game and I know or find out someone is stealing from me, I'll do something about it in character; and whether the person stealing from me is a PC or an NPC makes no difference whatsoever to what my character does next.
Ditto if I'm the one doing the stealing; if my character's dumb enough or unlucky enough to get caught I-as-player am well aware there's likely to be in-fiction consequences (and would, in all fairness, be annoyed were there not; if that lack of repercussions was based solely on the rationale that my character is a PC).
As both DM and player, I don't believe in "PC glow".