I agree actually. It could be.
Typically though, it’s not. People will blithely gloss over the hundreds of immersion breaking things that they don’t mind to focus on something they happen not to like and then pretend it’s about immersion instead of simply admitting that it just happens to be something they don’t like.
It’s the whole, I can believe six impossible things before breakfast but that seventh one? Whoo boy. And when it’s pointed out that the six other things are not immersion breaking somehow, you get accused of one true wayism or some other bit of nonsense.
I don't think that's quite fair. Especially in terms of "typically."
The reason that people can (and will) blithely ignore some things, but other things will bug them is because, as
@Thomas Shey noted, these are subjective preferences. This happens all the time, in all sorts of contexts.
To use the analogy I always go to- the suspension of disbelief operates differently for different people. There are a number of reasons for this- some people look to internal consistency (for example, some people were really bothered by how distances seemed to stop mattering in the last two seasons of Game of Thrones, others were not bothered by it). Some people bring their own experiences into it (a police officer watching a police procedural might have different issues than a regular person). Some people enjoy genre tropes, while other people find them annoying or unrealistic (one person's anime trope is another person's weird and unrealistic stylization). Some people enjoy more realism, some aren't bothered by a lack thereof (pew pew pew in space). And so on.
It's the same with games. Discussing "immersion" can be inherently fraught for two reasons-
1. It is always subjective. I might like a little science fiction in my fantasy (cue up Barrier Peaks), while another person might think that those streams should never be crossed because it pulls them out of the "fantasy feel." There's not a right or wrong, just a subjective feeling for what "works" for that person.
2. Because all the games are abstractions (and preferences are subjective), conversations are always going to be annoying. As you note, if you can believe six impossible things, why not a seventh? Well, because that's not how people work. It's like telling someone that loves Star Wars, "Look, you already accepted the
pew pew pew in space, so you can't argue against
anything that ever happens. There is absolutely nothing that can wreck your suspension of disbelief now! Muahahahahaha!" We often see versions of this play out in this sphere as well- "Look, once you've accepted there's magic, you can't disagree with anything. Also, since you haven't exactly modeled the economy, how can you say that you are playing a game with any semblance of reality? Check, and mate." That doesn't really add anything to the conversation.
Trying to tell someone that they are wrong (or arguing in bad faith) when they tell you that something doesn't work for them- that it wrecks their suspension of disbelief or immersion, is both (1) painfully easy because it's a fiction- and you will find many other examples that they will gloss over, but also (2) inherently improper, because you're telling them that their preferences, and what works for them, is wrong.