Is there a meaningful difference between a "rule" and a wide spread expectation?
Yes. Firstly, the term for the latter is generally "convention." Conventions are not required; they are simply a common practice done by many people. Sometimes conventions are for practical utility, e.g. righty-tighty, lefty-loosey. Sometimes they're done just for convenience, e.g. English acquired its indefinite article (a/an) from Anglo-Saxon speakers being too lazy to fully use the word "an"--which meant "one"--and dropping the "n" before words that started with consonants (e.g. "
an scip", "a ship", became "
a scip", but you wouldn't change "
an ācbēam", "an oak").
Great example: By convention, in many previous editions, 1st level characters would get maximized HP, and then roll for HP every level thereafter. The rules did not say to do this, but it was commonly done because folks understood that getting a really crappy roll for your 1st level HP led to a less fun experience in most contexts. This is not a convention in 5e, it is the actual rule of the game (and, further, you may choose to take the HD average rounded up, rather than rolling, which is also not a convention, it is actually what the rules say.)
A rule is how the game actually does tell you to play. "Do X." A convention is an adopted practice external to the rules, which you aren't technically
required to do. Non-gaming example: it is conventional, now, to orient maps so that north is at the top of the map. This convention has not always been the case. For example, ancient Egypt oriented their maps relative to the flow of the Nile, which is why "Upper Egypt" is in the south and "Lower Egypt" is in the north, as the upper/lower distinction is relative to whether you are nearer the headwaters or the delta of the Nile.
I don't know that there is in TTRPGs. I have to address deviations from either in pre-game discussions like Session 0, and I've seen people get more push back for not allowing feats, an optional rule, than for house ruling long rests, as an example.
A house rule is still a rule, so long as it is actually followed. It is not a rule that every table will use, of course, but if it is in fact a house
rule, it will be used consistently. Both of the things you are speaking of here are choosing to consistently use, or consistently not use, some specific rule. That is not the same as a convention.
So I think the DMG grants that power whether it's through a rule or through setting the expectation.
I disagree. If the things are actually rules, they should be followed consistently. If you don't want to follow them consistently, explicitly disclaim them with house rules--which are, themselves, actually rules--and make sure that the players
know those house-rules. It is dirty pool to have house-rules and only spring them on the player when they have done something that contradicts the house's rules that they didn't even know applied.
In both cases individual tables can override it, but in both cases they have to in order to avoid it. Many have stated they roll in the open for this very purpose. Because if they didn't players would assume they were fudging. Some, in the thread on that topic, even said they assume hidden rolls are being fudged.
I don't necessarily assume that hidden rolls are definitely 100% fudged, but it may invite the question, especially if other context clues are also present. Like if a DM typically rolls everything in the open, even Perception rolls (one of the
exceedingly rare cases where I think rolling in secret is ever even remotely warranted), and all of a sudden THIS roll is behind the screen...that's going to invite some questions, I should think. Likewise, if the DM pauses for a noticeable time after the roll is made, that could invite the question of whether they were re-evaluating. Etc.
In basically all of those cases, I would prefer that the DM level with me rather than lie about the result the die/dice indicated.
To me, I don't see much of a difference. The expectation is I, the DM, follow the rules - most of the time, kinda...
I do. A convention is just a frequent pattern, sometimes done for specific utility, sometimes done for convenience, sometimes simply because it is commonplace (e.g. whether a nation uses right-hand or left-hand drive
EDIT: I bet most players care more that you meet their expectations than you follow the letter of the rules. So maybe that's an indication.
But what expectations can they bring that are not playing the game you have offered to play? What expectation can they have when you claim that a monster crit, or that something was beyond their ability to detect, except that you actually used the rules honestly?
The rules provide the starting baseline. If those rules have ever and always "...unless the DM decided otherwise this time", then
they aren't rules, they're at very best suggestions and the only "rule" is the DM's whims.